Coach uses research-backed methods to measure career readiness
Our team developed a comprehensive impact model for AI-assisted career readiness in partnership with career development experts
What is career readiness?
Career readiness is the collection of attitudes, knowledge and competencies, and behaviors that contribute to success in an individual’s career journey 🚀
What we measure
The framework we’re using to define impact is a logic model based on Marciniak et al.’s career preparedness mode (see Career Preparedness Among Adolescents: A Review of Key Components and Directions for Future Research (2020)):
How we measure
Coach measures career readiness in two different ways:
CAAS-SF surveys provided at the start of learners’ engagement with the platform as well as after a period of time, to measure overall career readiness growth over time
Single-question mini-surveys, deployed after every activity completion, to measure the impact of specific activities on specific career readiness traits.
CAAS-SF Surveys
CAAS-SF, or the Career Adapt-Abilities Scale-Short Form, is a research-backed, validated, and widely used survey for career readiness (see Career Adapt-Abilities Scale–Short Form (CAAS-SF): Construction and Validation (2015))
To assess whether users have retained a longer-term behavioral change, we use the CAAS-SF survey at the beginning of their Coach journey and after a period of time for direct comparison:
Onboarding survey: 12-question survey (CAAS-SF) that will be included after sign up and welcome page. Questions appear 3 at a time. The survey will only be mandatory for learners within a classroom.
Follow-up survey: The same 12-question survey (CAAS-SF) will be sent again to the user once they complete 3 different activities and the user returns to Coach after 2 weeks. Questions appear 3 at a time.
Mini-Surveys
Since CAAS-SF measures attitude and behavioral changes that may occur on a longer timescale, we’re also implementing mini-surveys to assess the effectiveness of individual activities on building career readiness traits like interest, self-concept, self-efficacy, and belonging. These are single-question, yes / no surveys that appear after the first completion of an activity.
Bridging gaps in career readiness data
A complete framework for assessing career development outcomes does not exist, which makes it difficult to assess the effectiveness of career readiness efforts.
Historically, career readiness research has concentrated in two broad categories: career development theory (CDT) and intervention research.
Career development theory provides the theoretical underpinning for career readiness efforts. CDT answers the question: “What skills contribute to career readiness?” Examples include: Social Cognitive Career Theory (1994), Occupational Identity Development (2019), and A Career Education and Development Framework and measure for senior secondary school students (2023)
Intervention research is concerned with the practical application of career development theories, answering the question: “Does this intervention–always defined as a group of career development activities, like a career course–improve certain skills?” Examples include: Effects of a University Career Development Course on Career Decision-Making Self-Efficacy (2006), Increased Career Self-Efficacy Predicts College Students’ Motivation, and Course and Major Satisfaction (2013), and Career Self-Efficacy: A Mixed-Methods Study of an Underexplored Research Area for First-Generation, Low-Income, and Underrepresented College Students in a Comprehensive College Transition Program (2019)
What’s missing is data and research that maps career readiness skill development to specific career development activities. For example, we don’t know whether (or if) completing a career assessment (like the RIASEC) builds a specific career readiness trait like interest or self-concept.
This is where Coach can help us: by helping us collect much more specific, granular (and anonymized!) data that ultimately we use to map specific career development activities to specific career readiness traits.
Doing so unlocks a number of opportunities to improve (or build new) workforce readiness activities and programs.
Help
If you have more questions about how we measure career readiness, please reach out to us at help@aicareercoach.org