Title of presentation Mapping Professional Competencies to ... ?· Mapping Professional Competencies…

  • Published on
    04-Jun-2018

  • View
    212

  • Download
    0

Transcript

  • latrobe.edu.au CRICOS Provider 00115M

    Title of presentation Name of presenter Title of presenter School / Faculty / Division

    xx Month 201x

    Mapping Professional Competencies to Curriculum Bronwyn Carter Research Associate School of Public Health 20th September 2013

  • 2 La Trobe University

    Image area

    Demonstrating the extent to which curriculum meets competencies

    Problems in mapping trial Canada 2009

    No solutions identified, rater concordance not measured

    Canada-La Trobe Skills Project 2002-2009

    This project initial mapping results were unexpected

    Raters mapped independently

    Proportion of Agreement 44% 85%

    Achieving rater concordance not straightforward

    Changed focus to develop the mapping process using action research method

    Researchers La Trobe University Priscilla Robinson, Bronwyn Carter, Lesley Ervin Public Health Agency of Canada (PHAC) Jennifer Lowe, Elizabeth Wright

  • 3 La Trobe University

    Developing the mapping process

    Limited health sciences and education literature on the process for mapping competencies to curriculum

    Australian studies

    template designed for mapping generic skills to B Ed courses Macquarie University AARE Conference 2002

    mapped generic attributes required of IT course graduates to unit objectives QUT PhD 2004

    CDC funded study planned to map competencies for US epidemiologists to courses ASPH/CDC/CSTE 2008

    Inter rater reliability not measured, mapping processes not reported or not transferable

    This project aimed to map competencies developed in Australia and Canada to online courses while further developing the mapping process, findings published Journal Vocational Education and Training Vol. 65:3 2013

  • Course Material

    Competencies 30 Canadian 16 Australian

    Module Objectives 8 objectives

    Lesson 1 5 objectives

    Lesson 4 6 objectives

    Lesson 3 4 Objectives

    Lesson 2 6 Objectives

    Lesson 5 5 objectives

    Relationship between competencies and the module

  • 5 La Trobe University

    5

    The Mapping Process

    Revise Use the Rationale Document

    Compare and finalise results Document issues

    Compare Default options Discuss interpretation

    Decisions: Yes, No, ?Yes, ?No

  • 6 La Trobe University

    Good rater concordance achieved

    94%

    100%

    67%

    99%

    0%

    20%

    40%

    60%

    80%

    100%

    Initial Revised

    AustralianN=2205

    CanadianN=756

    85%

    99%

    44%

    88%

    0%

    20%

    40%

    60%

    80%

    100%

    Initial Revised

    AustralianN=525

    CanadianN=180

    Lesson 1 objectives Lesson 2 - 5 objectives

    Proportion of Agreement

  • 7 La Trobe University

    Mapping is a subjective process

    Variables identified as influencing mapping decisions:

    1. Interpretation of terms used in the wording of competencies and objectives

    C3.5 Demonstrate the ability to implement practice guidelines

    Similar problem found by PHAC, previous studies in Australia 2002 & Netherlands 2007

    Refer to PHAC glossary

    Discussion improved agreement (PHAC)

    2. Broad versus literal interpretations of competencies:

    Rater 1 58% Rater 2 63%

    3. Possible vested interests of raters/staff suggest involve students QUT 2004

  • 8 La Trobe University

    Mapping is a subjective process Variables identified as influencing mapping decisions:

    4. Level of proficiency assumed, not assessed

    4 point scale not addressed introductory intermediate advanced - used to assess courses University of Hawaii 1995

    meet spirit of competency (introductory level) used by 2 of 3 raters PHAC

    Dreyfus theory novice to expert recommended for use by public health graduates in self assessment by US Accreditation Council Graduate Medical Education 2008

    5. Rater knowledge of course content and familiarity with course, prior teaching and practice experience, one rater had developed curriculum and taught an online module

    6. Understanding of the implications of mapping curriculum to competencies and familiarity with the competencies increases while mapping

    may have contributed later in the project to fewer competencies rated as met

    83% Canadian rated as met 15% Australian rated as met mapped later

    7. Structure and phrasing of competencies - rater concordance better for Australian competencies, map to highest relevant element applied only to Australian

  • 9 La Trobe University

    Strategies and Tools

    Rationale for Yes Table

    Where there is discrepancy in results, rater who decided Yes completes table

    --> other rater accepts rationale or result defaults to No

    Resolves rater disagreement using a transparent process

    Full consensus was achieved

    Documents evidence for claims that objectives meet particular competencies

    Provides a basis for:

    review of the particular competencies and objectives

    Investigation of implications for curriculum mapping

    further development of the mapping process

  • 10 La Trobe University

    Strategies and Tools

    Issues Table

    Where a term needs clarification, document in Issues Table

    Records issues for later follow up

    Points for clarification may include meaning of particular terms or phrases in competencies or objectives

  • 11 La Trobe University

    Implications for curriculum development

    Interpretation of competencies needs to be sufficiently specific to inform curriculum development

    - 15% (16 of 105) Australian elements and

    - 83% (30 of 36) Canadian competencies considered to be met by the module

    Mapping to the lowest component of competencies is important to elements rather than Units

    - In 4 of 19 Australian Units and 3 of 7 Canadian Units only some competencies were considered met

    - US study 129 modules found few modules met all specific competencies within the broader competency Unit ASPH 2002

    Cross checking results within and across lessons assists in clarifying implications for curriculum development and aligning course material to competencies

  • 12 La Trobe University

    Click here to enter Title

    Click here to enter Text

    Click here to enter Text

    Click here to enter Text

    Click here to enter Text

    Australian Elements of Competence met by Lesson 1 objectives

  • 13 La Trobe University

    Australian Competencies integrated into learning objectives

    On completion of the Outbreak Investigation module participants will have developed competencies in the following Units of Competency:

    1. Monitor and evaluate population health data or indicators

    and may have developed competencies in the following Units of Competency:

    2. Analyse the quality of findings from a surveillance or screening program

    3. Plan a disease prevention/ control strategy

    4. Formulating and implementing a response to a public health emergency

    6. Mapping and analysing the environmental determinants that contribute to disease in a given community or population

    ..(continued).

  • 14 La Trobe University

    Australian Competencies integrated into learning objectives (continued)

    .And will be able to:

    Understand the goal, objectives and steps of an outbreak investigation;

    Be able to create and apply a case definition within the context of an outbreak investigation;

    Know the key elements of a standard investigation questionnaire;

    Be able to interpret descriptive data in terms of time, place and person characteristics;

    Be able to generate verifiable hypotheses on the source, mode of transmission, vehicle or vector of propagation, exposures or risk factors;

    Be able to identify appropriate control measures to halt an outbreak;

    Know the elements required for documenting an outbreak and how these may be used to prepare concise reports;

    Review the essential organizational and operational aspects of an outbreak investigation.

    ..(continued)..

  • 15 La Trobe University

    .On completion of Lesson 1 the participants will have developed competencies in:

    Monitoring and evaluating population health data or indicators (1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7)

    Analysing the quality of findings from a surveillance or screening program (2.1, 2.2)

    Formulating and implementing a response to a public health emergency (4.1, 4.2)

    Mapping and analysing the environmental determinants that contribute to disease in a given community or population (6.1, 6.2)

    And will be able to:

    Define the following terms: epidemic, outbreak, cluster and pandemic;

    Understand the goal and objectives of an outbreak investigation;

    List the conditions that enable outbreaks to occur;

    List the main reasons for investigating a presumed outbreak;

    List the main steps of an outbreak investigation and

    Apply these competencies, knowledge and skills to their place of work.

  • 16 La Trobe University

    Recommendations for future mapping

    Further trial strategies and tools:

    the Question mark and Default system

    Rationale for Yes

    Issues Table

    Map to lowest relevant element (for competencies in hierarchy)

    Consider possible rater bias, use independent raters

    At least 2 raters, ideally 3 raters

    Trial proficiency ratings at least 2 levels

    Cross check results to clarify implications for curriculum development

    Trial suitable data base software

    Revise mapping outcomes for Outbreak Investigation and Management module

  • Bibliography

    Australian Network of Academic Public Health Institutions. Foundation Competencies for Master of Public Health Graduates in Australia. ANAPHI; 2009. Dato V, Fertman C, Pistella C. A Capacity Mapping Approach To Public Health Training Resources. Public Health Reports2002;117:20-8. Leach D. Competencies: From deconstruction to reconstruction and back again, lessons learned. American Journal of Public Health2008;98:1562-4. Loos G. Minimum Competencies for Public Health Personnel. Asia-Pacific Journal of Public Health1995;8:195-200. McNutt L, Furner S, Moser M, McGean Weist E. Applied Epidemiology Competencies for Governmental Public Health Agencies: Mapping Current Curriculum and the Development of New Curriculum. Public Health Reports 2008;123:13-8. Public Health Agency of Canada. Core Competencies for Public Health in Canada 2007. Available from: http://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.php. Piloting of the competency mapping process to the skills online module - a synopsis. Skills Enhancement Team. Public Health Agency of Canada. 2009. Process guide for competency mapping. Skills Enhancement Team. Public Health Agency of Canada. 2009. Snoke R. Generic Attributes of Australian Information Systems Graduates: An Empirical Study: Queensland University of Technology; 2004. Stoof A, R M, van Merrienboer J. Web-based support for constructing competence maps: design and formative evaluation. Educational Technology, Research and Development 2007;55(4):347-67. Sumsion J, Goodfellow J. Identifying Generic Skills Through Curriculum Mapping: A Critical Evaluation. Annual Conference of the Australian Association for Research in Education; Brisbane 2002. Graphics Novice to expert Charlotte Wolff http://portal.ou.nl/en/web/cwo/blog/-/blogs/the-rare-and-arational-expert-teacher

    http://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.phphttp://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.phphttp://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.phphttp://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.phphttp://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.phphttp://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.phphttp://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.phphttp://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.phphttp://www.phac-aspc.gc.ca/ccph-cesp/stmts-enon-eng.php

  • Thank you

    latrobe.edu.au CRICOS Provider 00115M

    Curriculum mapping: not as straightforward as it sounds. Journal of Vocational Education & Training, 2013 Vol.65 No. 3 309-318. Lesley Ervin, Bronwyn Carter & Priscilla Robinson

    Further information:

    Bronwyn.Carter@latrobe.edu.au

    Priscilla.Robinson@latrobe.edu.au

    mailto:Bronwyn.Carter@latrobe.edu.aumailto:Priscilla.Robinson@latrobe.edu.au

Recommended

View more >