November 30-December 1, 2003
Toronto, Ontario

************************************************************************************

Executive Summary

The first annual Strategic Training Initiative in Health Research (STIHR) Workshop was held from November 30 - December 1, 2003 in Toronto.  The purpose of the meeting was to strengthen linkages between people and programs, to discuss challenges and identify solutions to them, and to present and hear the response to the STIHR evaluation framework developed by CIHR. 

Small group and plenary discussions focused on eight main topics:

• Recruitment strategies
• Distance education/Web-based learning
• The capacity of host institutions to support interdisciplinary training
• Interdisciplinary research and training
• Curriculum/program structure
• Mentoring
• Involving clinicians and clinician trainees
• Organizational roles and relationships

Each small group discussed the ideal state, challenges, and next steps associated with its respective topic. Their principal recommendations are summarized below:

Recruitment Strategies

1. Each program should develop a promotional slide or splash page to include in every meeting and conference presentation delivered by trainees, principal investigators, and mentors.
2. CIHR should take a leadership role in marketing the research programs. To help students easily find and compare program information, CIHR should create a simplified, fast Web site that includes links to individual STIHR programs. A Frequently Asked Questions page that addresses common questions should also be developed for the site.
3. Integrate questions about best recruitment practices in the review process. Circulate best practices information to individual programs.
4. Sponsor "research days" at which trainees can expose other students to the interdisciplinary curriculum.

Institutional Support

5. Tie training grant funds to evidence of institutional support. Follow up after grants are awarded.
6. Continue to require deans of graduate studies to be involved to some degree in the grant application process.
7. Arrange for senior CIHR staff, such as Alan Bernstein, to meet regularly with deans to communicate the goals of STIHR and the importance of interdisciplinary training. Encourage program directors and principal investigators to initiate dialogue with the decision makers in their institutions. Send a delegation made up of representatives from CIHR and the training programs to deliver a promotional presentation at the annual meeting of deans of graduate studies.
8. Create shared faculty appointments, possibly funded by CIHR, that are supported by additional stipends for directors.

Interdisciplinary Research and Training

9. Review CIHR policies, procedures and criteria to see if they work against transdisciplinary research.
10. Encourage university departments to recognize the value of interdisciplinary teaching when awarding tenure, merit, and promotions.
11. Demonstrate interdisciplinary interaction by having mentors from different disciplines share teaching responsibilities.

Curriculum and Program Structure

12. With CIHR support, develop a course or module on transdisciplinary research. CIHR could conduct a needs assessment, issue an RFP, and hire an educational consultant to develop the content.
13. Provide a Web-based clearinghouse, separate from the CIHR Web site, where programs can share templates for documents such as brochures, forms, and reports.
14. To reduce duplication of effort, establish a CIHR working group to address common program requirements such as trainee evaluation criteria.

Web-Based/Distance Learning

15. Identify resources that explain how to translate courses for use in a Web-based learning environment. Create an electronic library that all programs can access.
16. Identify instructor competencies needed to teach on-line.
17. Regulate and standardize Web content through CIHR. Draft contracts and agreements to protect CIHR Web content from copyright challenges.

Mentoring

18. Have CIHR compile resources on effective mentoring and post them on the CIHR Web site. CIHR should be responsible for marketing the site and for responding to mentor requests for information.
19. Train PhD and post-doctoral candidates in mentoring within their STIHR programs.

Involving Clinicians and Clinician Trainees

20. Provide more support and recognition for mentors, e.g. hire senior research associates to take on some of the load.
21. Consider alternative funding plans, such as block training grants, as a way to recruit or keep scientists. CIHR could take a leadership role by requiring training programs to involve clinicians as a grant requirement.
22. Encourage a more flexible definition of clinician-scientist that might allow practitioners to spend a short time in research and then if they like it, to go on.
23. Include both basic scientists and practitioners in the review process.
24. With CIHR taking the lead, encourage medical schools, sub-specialties, the Royal College of Physicians and Surgeons, and graduate education departments to create a clear career path for clinician-scientists.

Organizational Roles and Relationships

25. Develop a plan, perhaps under the leadership of CIHR, for moving towards harmonization across institutions.
26. Invite the deans of graduate studies and health research organizations to meetings like the STIHR workshop so they can see what STIHR is trying to accomplish.
27. Have principal investigators work within their own institutional cultures.
28. Involve training program leaders with institute advisory boards.

Workshop Objectives

The Canadian Institutes of Health Research (CIHR) and its partners (see Appendix II) launched the Strategic Training Initiative in Health Research (STIHR) to build Canada's health research capacity and to promote an interdisciplinary model among health researchers. The first call for applications resulted in a pool of 51 funded transdisciplinary programs in March 2002. A further 35 projects received funding in April 2003.

The CIHR provides a Web site for recruitment ads, print-ready recruitment ads that can be tailored for individual programs, and a list serve on which participants are encouraged to post questions and share information.

From November 30-December 1, 2003, program coordinators, principal investigators, trainees, and funding partners convened in Toronto for the first annual Strategic Training Initiative in Health Research Workshop. The purpose of the meeting was to strengthen linkages between people and programs, to discuss challenges and identify solutions to them, and to hear the response to the STIHR evaluation framework developed by CIHR. 

Small group and plenary discussions focused on eight main topics: recruitment strategies; mentoring; the capacity of host institutions to support interdisciplinary training; distance education/Web-based learning; curriculum/program structure; involving clinicians and clinician trainees; interdisciplinary research and training; and the roles of the CIHR secretariat, CIHR Institutes, training programs and partners.

Expectations

The workshop began with a discussion among participants about what they hoped to achieve during the workshop. They identified the following objectives:

• Discuss ways to increase university support for the STIHR program through matching awards, release time for principal investigators and coordinators, and recognition of the contribution made by mentors.
• Explore strategies for implementing transdisicplinary programs in institutions that are structured along disciplinary lines.
• Share ideas for recruiting the best young researchers from Canada and abroad, including established clinicians.
• Establish and deepen connections between programs, between trainees, and between researchers.
• Take away concrete ideas for efficiently managing program resources and allocating funding.
• Clarify the STIHR evaluation process and learn about CIHR's long-term funding intentions.
• Share ideas for evaluating trainees who are studying in a transdisciplinary system in which they may take courses in different faculties and at different universities.
• Discuss the big picture-how do we foster synergy between the 86 funded programs? Are the partners satisfied that their investment is worthwhile?

Summary of Opening Remarks

Dr. Alan Bernstein, President, Canadian Institutes of Health Research

The hot health research areas today are multidisciplinary because scholarship in fields ranging from the biological sciences, to humanities, to social sciences, to engineering, is relevant to health. "Bioethics", "epidemiology", and "health law" are all fusion terms that reflect the interdisciplinary thrust of the CIHR Strategic Training Initiative. The STIHR is about young people, about excellence, about partnerships, and about embracing every aspect of health care.

After widespread consultation, CIHR has launched a strategic plan called BLUEPRINT which sets out five broad directions for the next four years: to strengthen excellence in health research; to identify emerging health challenges such as SARS and determine the national platforms needed to meet them; to support a balanced research agenda that aims to understand disease AND treat it; to harness research to improve the health of vulnerable populations; and to support the strengthening of Canada's health system through innovative programs that bring together the creators and users of new knowledge. We expect to see a deepening commitment to research from the next government; Paul Martin has indicated that he supports this direction and will be an ongoing investor.

Among our tasks as health researchers is the duty to engage the public in health research by learning how to communicate what we are doing. We at CIHR are doing all we can to make Canada the "place to be" for health research. But we cannot do it alone. Our greatest asset in this challenge is you. I encourage you to speak to your local members of Parliament and let them know of the excellent work that is going on across Canada thanks to federal government support. This action on your part is of immense help in our task of trying to secure the financial growth we need to support your work.

A. Lessons Learned: Panel Presentation

Tomorrow's Researchers Cardiac Health Program
Dr. Evangelis Michelakis, University of Alberta

The Program
TORCH is an integrated program at the Universities of Alberta and Calgary dedicated to preparing Canada's next generation of transdisciplinary cardiovascular health research leaders. The program incorporates case-based learning, a journal club, seminars, and workshops.

The Challenge
Our main challenge has been to bridge two remote sites, the University of Alberta in Edmonton and the University of Calgary in Calgary, which are equal partners in the grant. The program has successfully devised sophisticated, interactive technologies to connect the sites. A second challenge is to engage clinicians and clinician trainees in a commitment to research. In this we have been less successful.

Strategies
We designed a sophisticated, interactive, password protected desktop with links to journal and textbook databases, on-line help, a PDA download, reading assignments, and presentations. A desktop calendar notes training sessions, topics, and important dates. Presentations and slides are archived and can be accessed for viewing and discussion. Trainees rate each session and presentation on-line. Their evaluations are then analyzed and posted for mentor viewing.

Every Thursday afternoon, trainees and mentors at both sites participate in a videoconference. Each site takes a turn as chair. Three times a year we hold a workshop for the entire team. Either the group from one site visits the other site or we all travel to a third location. We have learned that chartering a small plane is a simple and cost effective way to bring the two teams together; the fact that our program coordinator is a licensed pilot is a bonus!

Health Care, Technology and Place, University of Toronto
Kelly Murphy, Deputy Director

The Program
New technologies are making it possible to deliver health care in a range of new settings-where people live, work, shop, surf the net, and go to school. Moreover, new technologies are transforming traditional health care settings, like hospitals and clinics. The objective of HCTP is to prepare new and established researchers to understand, explain, and improve health care outcomes associated with changing health care settings.

The Challenge
To make our integrated program comprehensible in an administrative language that is based on divisions, to maximize efficiency by sharing resources with other research centres on campus, and to enable faculty to collaborate on interdisciplinary projects.

Strategies
Administrative structure: We launched our training program as an Ontario Council on Graduate Studies-approved collaborative program, which is a format that is recognized at the University of Toronto. While the process was intensive, requiring approval from every academic division, the effort has paid off. Departments now see themselves as stakeholders in HCTP. Moreover, accreditation has facilitated recruitment. The departments work with us to recruit students into HCTP and they widely profile our activities. 

Efficiency: We created a partnership of six research and teaching centres at the U of T that explore the application of health and information technologies. All partners benefit from this integration. For example, each year the HCTP builds a team-taught course for students at U of T and collaborating universities. Beginning in January, these meetings will take place in networked classrooms, courtesy of one of our partners, and Web cast to participants in other locations by another partner. In effect, we supply the students, the teachers, the curriculum, and a study sample, and the other groups provide the facilities, the technology, and the program evaluation expertise.

Infrastructural support for interdisciplinary projects: Last year we applied for and won a CIHR Interdisciplinary Capacity Enhancement Team (ICE) grant, to complement the Training Program grant and to support faculty research and peer to peer mentoring. We use ICE money to pay for research assistants, seed grants, collaborative writing and grant preparation, program evaluation and strategic planning. To date we have held two conferences and have four subteams engaged in interdisciplinary research. Each of these teams involves established faculty, junior faculty, and most importantly, at least one fellow from our training program as a co-investigator.

Pain in Child Health
Dr. Patrick McGrath, Dalhousie University

The Program
Pain research is inherently interdisciplinary. We are trying to make it more so by developing a community of scholars in a program that integrates basic, clinical, psychosocial, and health services research. The program has six principal investigators located in five centres across Canada. Trainees were drawn from nursing, psychology, pharmacy, neuroscience, pediatrics and neonatology.

Challenges
We found some CIHR policies to be constraining (we ignore the requirement to include all participating names on papers because they take up too much room) or silly (no funding for cell phones). There is some shortage of basic science trainees and mentors and we have had limited success in cross-discipline supervision. Although the host institutions are enthusiastic about the program, they have not translated their support into space or money.

Strategies
The principal researchers share a vision and have worked together previously on other grants and projects. In addition, we have an excellent coordinator who has a background in adult education and distance learning.

We organize a regular Web-based lab meeting with sites across Canada as well as one in Australia. We also hold a monthly management meeting via conference call. We publish a newsletter, sponsor a list serve, and host an international speaker series. We also organize institutes on mentor training. These are always linked to international or national meetings so our trainees can attend both. We recruit by word of mouth. Our senior researchers all have many students and have been able to attract outstanding trainees.

B. Group Discussions

Participants broke into groups to discuss those topics identified prior to the workshop as key issues. They first envisioned an ideal state, then considered the barriers to achieving that state, and finally proposed solutions to address the challenges.

1. Recruitment Strategies

The Ideal State
Effective marketing will attract a sufficient number of excellent, focused applicants, from Canada and abroad, to meet the number of positions in our programs. Recruitment will maintain a balance between excellence and transdisciplinary strategic needs. There will be standard evaluation criteria across programs. Trainees will be able to depend on competitive and adequate stipends for the duration of their programs.

Barriers
Student Motives
Too often we find that students use research as a place to park while they wait to get into medical school. We want to promote research as a career.

Marketing Know-how and Program Visibility
The 86 grantee groups are duplicating effort by trying to market their programs individually, without marketing expertise. The STIHR program as a whole is not well known to students and post-doctoral students.

One program reported that an analysis of various marketing strategies (e.g. ads, personal contact) showed that recruiters most successfully attracted the best applicants through personal appeals, such as promoting training opportunities during presentations at conferences.

Stipend and Tuition Differentials
The ability to provide adequate stipends for clinicians is an issue (although all that CIHR requires is a minimum stipend; the only limiting factor is the size of the grant). Tuition rates differ for Canadian and foreign students which discourages international participation. Teams also need standard agreements for sharing students. How do we maintain a balance between excellence and interdisciplinarity in recruitment, bearing in mind the students' different funding requirements?

Institutional Commitment
Once funded, there is no pressure on the universities for institutional commitment.

Next Steps

• Students are the best ambassadors for STIHR programs. All students and investigators should include a promotional slide in every conference or meeting presentation.
• CIHR should take a leadership role and centrally market research programs on a simplified CIHR Web site. The site should include links to each research training program and a Frequently Asked Questions page to address common questions. On a more general level, CIHR should promote health science research as a career.
• CIHR should foster institutional support for the program and for researchers through regular contact with deans. It should also consider tying funding to evidence of university support. For example, CIHR could require that the university provide a space for the coordinator as a condition of receiving funding. Principal investigators could join forces to lobby administrators for more support. 
• CIHR can strike a working group to address common questions of procedure, such as trainee evaluation criteria, to avoid having every program devote resources to work out similar solutions to common problems.

Recommendations

• Each program should develop a promotional slide or splash page to include in every meeting and conference presentation delivered by trainees, principal investigators, and mentors.
• CIHR should take a leadership role in marketing the research programs. To enable students to easily find and compare program information, CIHR should create a simplified, fast Web site that includes links to individual STIHR programs. A Frequently Asked Questions page that addresses common questions should also be developed for the site.
• Integrate questions about best recruitment practices in the review process. Circulate best practices information to individual programs.
• Promote health research as a career with a CIHR-produced poster and Web page. Reach students as early as the high school level.
• Sponsor "research days" at which students can expose other students to the interdisciplinary curriculum.
• CIHR should take the lead in encouraging medical schools, sub-specialties, the Royal College of Physicians and Surgeons, and graduate education departments to create a clear career path for clinician-scientists.

2.  The Capacity of Host Institutions to Support Interdisciplinary Training

The Ideal State
Universities and hospitals will be restructured to facilitate interdisciplinary training: a reciprocal credit system will allow trainees to easily participate in courses at different universities and in different faculties; institutions will share common ethics, finance, and human resource processes; universities will recognize and reward interdisciplinary activities in tenure consideration; interdisciplinary faculty positions will exist; equal resources will be available for equal work across disciplines; and value added will be measurable.

Barriers
Organizational structures
A rigid departmental university and hospital structure is difficult to overcome.

Lack of Suitable Publication Vehicles
Like institutions, academic publications are normally subject specific. There are few publications or other vehicles for disseminating multidisciplinary research. In addition, multidisciplinary research involves larger author sets making it difficult to fairly evaluate the contribution of each author. This is especially troublesome in the humanities where only single authorship is recognized. Traditional measures of impact, such as index citations, are irrelevant; new metrics are required.

Next Steps

• Reinforce the value-added component of the programs by issuing CIHR certificates and credits.
• Make grant renewal contingent on institutional support.
• Have CIHR meet regularly with deans to communicate the importance of interdisciplinary training. The best way to capture the deans' attention is to show them that something innovative is going on at another university.
• Devise new ways for publications and academic bodies to review and evaluate research to take interdisciplinary research into account.
• Recognize that industry will partner more readily with interdisciplinary research because it matches corporate objectives. Interdisciplinary researchers can leverage this knowledge.

Recommendations

• Tie training grant funds to evidence of institutional support.
• Continue to require deans of graduate studies to be involved to some degree in the grant application process.
• Arrange for senior CIHR staff to meet regularly with deans to communicate the goals of the STIHR and the importance of interdisciplinary training. Encourage program directors and principal investigators to initiate dialogue with the decision makers in their institutions. Send a delegation that includes representatives from CIHR and the programs to deliver a promotional presentation at the annual meeting of deans of graduate studies.
• Design a value-added recognition mechanism, such as national standards, for collaborative programs. Formally recognize the value of participating in a STIHR program with a CIHR certificate.
• Create shared faculty appointments, possibly funded by CIHR, that are supported by additional stipends for directors.
• Invite deans of graduate studies and health research organizations to STIHR workshops so they see what we are trying to accomplish.
• Review CIHR policies, procedures and criteria to see if they work against transdisciplinary research.

 3. Interdisciplinary Research and Training

The Ideal State
Researchers who think at a complex level in multiple areas will work within a culture of interdisciplinary research. They will either begin as oaks, who branch out once they have a strong foundation in one discipline, or as ferns, who have branching areas of interest from the start.

Barriers
Language
Different disciplines develop their own languages and jargon to communicate about highly specialized knowledge. In the interdisciplinary setting, language can be a barrier to learning and collaboration. While specialized language may impede understanding, simplified language can make it difficult for mentors and trainees to present complex ideas. Is it possible to use high school language to express PhD content?

On the other hand, putting together people from different disciplines forces them to either learn new languages or to become better communicators. The ability to explain complex ideas with clarity is valuable for teaching undergraduates and international students and for explaining our work to non-researchers, such as journalists.

The University Structure
In theory, interdisciplinary scholars should be the most attractive university hires because they bring a broad view to research. In reality, universities are rigid, traditional institutions structured along departmental lines.

Next Steps/Recommendations

• Develop a course or module for all trainees and mentors about transdisciplinary research and how to carry it out. CIHR could be involved by conducting a needs assessment, issuing an RFP, and hiring an educational consultant to develop the content.
• Encourage university departments to recognize the value of interdisciplinary teaching when awarding tenure, merit, and promotions.
• Have mentors from different disciplines share teaching responsibilities so that they can demonstrate interdisciplinary interaction.
• Create shared faculty appointments, possibly funded by CIHR, that are supported by additional stipends for directors.
• Encourage transdisciplinary training in the CIHR review process.

4. Curriculum and Program Structure

The Ideal State
There will be an innovative curriculum and program, recognized by participating institutions and the workplace, which has international participation. It will offer students opportunities for internships and employment, involve them in teaching and training, and invite them to contribute to the adjustment, change, and growth of the program. Resources will exist that permit discussion and resource sharing among all 86 STIHR programs. All participants will have access to an extensive virtual library. 

Barriers
Institutional Receptivity
The transdisciplinary model differs from traditionally structured advanced courses which are designed in advance, then delivered. Transdisciplinary education must be flexible and open to feedback in order to meet the needs of diverse participants. At the same time, it must maintain excellence and depth of scholarship. Moreover, universities feel pressure to shorten programs and graduate students faster. Transdisciplinary education adds complexity, making it more difficult to meet the demand for speed.

Next Steps
CIHR can play a facilitating role in developing models of transdisciplinary education that maintain excellence. The organizations that have successful transdisciplinary experience should serve as role models by engaging in collaborative research.

Recommendations

• With CIHR support, develop a course or module on transdisciplinary research. CIHR would conduct a needs assessment, issue an RFP, and hire an educational consultant to develop the content.
• Provide a Web-based clearinghouse, separate from the CIHR Web site, where programs can share templates for documents such as brochures, forms, and reports.
• To reduce duplication of effort, establish a CIHR working group to address common program requirements, such as trainee evaluation criteria.

5. Distance Learning / Web-based learning
 
The Ideal State
Training programs will be able to expand their scope and influence through the use of on-line resources. Web-based resources will create a high level of interaction across geographical barriers. Trainees will teach and study on-line. Programs will easily share common courses on topics such as grant writing and ethics.

Barriers
Resources
The cost of developing and launching secure Web resources is high and the quality can be uneven.

Copyright Protection
Programs have to protect copyrighted material and intellectual property.

Teaching and Learning
On-line teaching and course development requires excellent teachers who can break course material into modules. Not all teachers have the skills to prepare and deliver on-line courses. While Web-based study bridges geographical barriers, trainees and mentors may lose the creative, unexpected learning that can take place when people sit in a room together.

Next Steps

• Create a common resource to consolidate information that all programs can access so that 86 sites are not spending resources to develop similar on-line courses e.g. how to write a grant application.
• Draft contracts and agreements to protect against copyright challenges. Ensure that journals, authors, and associated organizations grant permission for posting material on-line.
• Regulate and standardize Web content through CIHR.
• Ensure that information and presentations are formatted for distance learning applications. Hire experts to design effective course formats.
• Verify an instructor's ability to deliver instruction on-line.

Recommendations

• Identify resources that explain how to translate courses for use in a Web-based learning environment. Create an electronic library that all programs can access.
• Identify instructor competencies needed to teach on-line.
• Regulate and standardize Web content through CIHR. Draft contracts and agreements to protect distance learning sites from copyright challenges. 

6. Involving Clinicians and Clinician Trainees

The Ideal State
Programs will easily recruit, engage and retain clinical trainees and practitioners who are in the earliest stages of their education. Clinicians will have protected time for research and will be supported by flexible regulatory bodies and their own departments. Their role in research will be valued and rewarded with concrete credits such as CME points and co-authorships.

Barriers
Career Trajectory
Trainees must see from the earliest stages in their education that there will be no road blocks in their career path if they spend time engaged in research. Research clinicians must see a real career trajectory; if they bump up against barriers when they apply for research grants, for instance, they will get frustrated.

Rewards
Clinician-scientists lack a community. Their role as mentors is not recognized or rewarded by either basic scientists or clinicians. And mentoring takes time. As long as clinicians work within a fee-for-service salary structure, the mentoring role will costs them money.

Time
Even when they have a lab, clinicians have no time to conduct research.

Next Steps/Recommendations

• More support and recognition for mentors, i.e. hire senior research associates to take on some of the load.
• Alternative funding plans, such as block training grants, may be a good way to recruit and keep scientists. CIHR could take a leadership role by requiring training programs, as a grant requirement, to involve clinicians.
• Encourage a more flexible definition of  clinician-scientist that perhaps allows practitioners to spend a short time in research and then if they like it, to go on.
• Rethink the view that all practitioners are clinicians; people from the community who do research are just as valuable.
• Include both basic scientists and practitioners in the review process.

7. Mentoring

The Ideal State
A mentor is a friend and champion who knows the trainees and understands their goals. Mentors have a thorough knowledge of their subject and are able to integrate relevant information from different sources. They listen well and have the flexibility to adapt to a variety of students. Their capacity to guide research topics is especially important for interdisciplinary students.

Barriers
Territorial Boundaries and Power Issues
Students receive conflicting advice from mentors, committee chairs, and departments. It is difficult for a student to reject advice from a committee that will later judge his or her thesis.

Recognition
There is no built-in "win" for mentors in terms of pay, time, or credit. They take on mentoring in addition to their other duties.

Training
Are mentors born or trained? While mentoring comes naturally to some, others need better skills.

Next Steps

• Train mentors and trainees in mentoring techniques.
• Fully explain the mentoring role to potential mentors before they begin to work with trainees.
• Appoint a learner advocate or ombudsman to defend trainee interests.
• Reward mentors with protected time, money, and credit.

Recommendations

• CIHR should compile resources about effective mentoring and post them on the CIHR Web site. CIHR should be responsible for marketing the site and responding to mentor requests for information.
• Train PhD and post-doctoral candidates in mentoring within their STIHR programs. 

8. Organizational Roles and Relationships

The Ideal State
There will be harmonization across institutions. Partners, universities, hospitals and the CIHR will work across institutions in support of a shared vision.

Barriers
Traditional Reward Systems
Tenure and promotion policies conflict with what we are doing.

Absence of Coordinating Structures
Differing fee structures make it difficult for students to take courses at different schools. Leadership is needed to tackle universal issues. Although this program is creating a new generation of researchers, the energy and work required to succeed is not valued everywhere, a fact that is underlined when students face review committees and are not always rewarded.

Next Steps/Recommendations

• Map out a plan for moving towards harmonization, perhaps moved forward by CIHR.
• Invite deans of graduate studies and health research organizations to meetings like this workshop so that they can see what we are trying to accomplish. Require deans to be involved to some degree in the grant application process.
• Principal investigators can work within their own institutional cultures.
• Involve training program leaders with institute advisory boards.

9. Trainee Perspective

The trainees that had been invited to the workshop met to discuss the STIHR experience and presented their views on the program's strengths and the areas that need improvement.

Strengths
Students are exposed to multiple mentors, institutions, trainees, and partners in industry, which gives them opportunities to network for future collaborations. They see a strong mix of disciplines and the possibility of developing innovative programs that do not fit within existing institutional or departmental boundaries. Exposure to a variety of disciplines has introduced them to models they can incorporate into their own research. They are learning to mentor and they value Web-based learning.

Areas for Improvement

• Some training programs are moving forward slowly and will not have reached their potential in terms of curriculum content by the time these students graduate.
• National/international coordination is difficult.
• Because of the diversity of stipend levels, trainees are unsure of whether to shop around for the best pay or the best training opportunity.
• Broad research topics within the interdisciplinary model make it difficult to find common curriculum content.
• Trainees need more transparent evaluation criteria and more regular feedback from their principal investigators and mentors so they know whether they are developing properly as future researchers.
• To promote research as a career, training opportunities should be offered at all levels, beginning with undergraduates.
• Knowledge management could be stronger. Why reinvent the wheel by designing new courses that all programs need, such as courses on ethics?
• Training must be balanced to give trainees working outside of their fields the time to adjust to new language and terminology. Alternatively, principal investigators and mentors could develop some common language and terminology.

D. The Evaluation Framework
Performance Measurement and Evaluation Framework

Feedback
Content

• Give us the opportunity to tell CIHR the questions we want answered.
• Let the programs determine milestones and explain how they track what they are doing.
• The 70/30 split as an evaluation criterion is a big problem in meeting our objectives. Ask us how we budget to meet the 70/30 split. Major questions remain about managing the split when students have outside awards. We are unable to compare international trainees who are not eligible for funding outside the STIHR.
• There should be room to highlight factors that are unique to particular programs and at the same time to consider what the programs have in common.
• Risk management is a critical factor. The environment (funders, university decision makers) also needs to be considered in the evaluation.
• Employ quantitative and qualitative methods.
• Watch for biases in comparisons between groups of trainees.
• Ensure a distinction between tracking data and evaluation data.
• Strategic vs. excellence is a false dichotomy. Instead we need to think of transdisciplinary vs. disciplinary training.
• Do not track transdisciplinarity at the expense of excellence. Transdisciplinary research is fundamentally different from other research; the evaluation must respect the difference
•  Some evaluation criteria do not match the structure of particular programs, e.g. number of students reached. Some will find it difficult to alter their preferred mode of measuring success to fit an "academic" evaluation framework. There is a risk of distorting reporting or shaping programs to meet the evaluation criteria, similar to " teaching to the test". It became clear during the workshop that there are significant differences in program format. A section for descriptive explanation in the progress report would address the unique goals of each program.
• How will criteria be weighted?  Will programs be compared to each other and if so, how?
•  Knowledge transfer needs to be a criterion.
• Programs should be evaluated against the original objectives set out in the original grant applications.
• How will the data collected be used to make decisions about program funding
•  Mid-term review should be based on serious failures, not detailed point-by-point evaluations.
• What have we done to help our partners?
• There are no questions about how many trainees applied to the program but were rejected.
• Logic model provides guidelines to programs to ensure that they meet goals.

Time Frame

•  Set a realistic time frame. It will take time, effort, and resources to fulfill the reporting structure required.
• Don't make the mid-term review too resource intensive.
• The October timing of annual reports conflicts with operating grant deadlines.
• Separate the deadlines to ensure feedback from the annual report. Schedule the mid-term evaluation at four years to reflect doctoral students, and keep the summative evaluation at the end. A mid-term review should be a built-in from the second annual report.

Program Needs

• Better definitions of "transdisciplinary" and an operational definition of a trainee.
• Custom software to allow consistency between programs and to allow programs to divert some of the reporting responsibilities to trainees.
• Clearer explanations of the criteria and how to measure them (e.g. How do you define "high quality trainees" in Table 7; or "top 10% of the cohort"?)
• Key performance indicators and criteria so we can collect information for individual trainees on annual report forms.
• As much information as possible as soon as possible to prepare mid-term reports.
• Web-based forms, but only if the CIHR Web site becomes faster, otherwise evaluation will be unmanageable.
• Deadlines that don't conflict with grant application deadlines. Feedback from annual progress reports should be available to programs in a timely fashion, well before we prepare the midterm report.
• Simplified logic model applicable for students and mentors
• A dedicated person at CIHR to solve problems that arise for the coordinators. A direct line to Karl and Angus as we prepare our progress reports. Designate a period of days well before the deadline when CIHR staff will be available for prompt responses to questions.
• Struggling programs need help from more advanced or successful programs. General feedback about other programs, such as a summary of their strengths and weaknesses, would be helpful.
• Guidelines from CIHR regarding which criteria they consider paramount. Science is a continuum. Demonstrating transdisciplinarity may require great patience.

Final Remarks
Mark Bisby, Vice-President, Research Portfolio, CIHR

From CIHR's point of view, the first annual STIHR Workshop was very valuable. We received wonderful feedback about the program. And the CIHR has heard you. We have heard what you think of eligible expenses. We have heard a lot about the 30/70 rule. We may not be able to address the issue right away but we have heard about it and will take a look. We will look at the small irritants. Decisions about expenses like cell phones were developed at the outset through educated guesses-we will certainly revisit them.

We know that we have to engage the universities and that we can't leave this up to you. We do have regular meetings with universities and now we know what topics we need to raise.

We also know that hearing from you annually is not enough. We need to have an ongoing forum. We may put in place a working group so we can deal on an ongoing basis with issues that arise.

Thank you for your participation. We hope to see you all at the next meeting.
 
Appendix I: Defining Uni-, Multi-, Inter-, and Trans-Disciplinary

Requests for definitions of transdisciplinary, multidisciplinary, and interdisciplinary came up frequently during the STIHR workshop. The following descriptions may be helpful:

1) The requests for applications for the 2002 and 2003 Training Program Grant competitions defines interdisciplinary as "the ability to analyze, synthesize and harmonize links between disciplines into a coordinated and coherent whole."

2) Other sources suggest viewing research collaboration in terms of the disciplines involved and the way in which the researchers interact.   Unidisciplinary collaboration encompasses research interaction within a single discipline. One might expect that it will lead to more effectively solving a problem or answering a question defined by that discipline's paradigm. Multidisciplinary collaboration brings together researchers from different disciplines who individually address specific aspects of the research challenge using the perspectives and methods of their respective disciplines. Their findings are drawn together but not necessarily integrated. Interdisciplinary collaboration also brings together researchers from a variety of disciplines. However, they work towards a common understanding of the research problem, attempting to share disciplinary perceptions. While each researcher tends to use the theories and approaches from his or her discipline, there is a concerted effort to collect and integrate findings from all members of the team. Transdisciplinary collaboration begins with a recognition that the problem to be addressed is part of a complex system and that its true nature cannot be perceived through the lens of any one discipline.  The research may be held together by a common ideological framework, such as structuralism, semiotics, general systems theory, or postmodernism. The researchers make a considerable effort to understand the problem and the other disciplinary perspectives that it calls for. Collaboration continues in the development and implementation of research projects and the interpretation of results.
 
Appendix II: Partner Organizations in the Strategic Training Initiative in Health Research (STIHR)

Alberta Cancer Board
Alberta Heritage Foundation for Medical Research
Canadian Health Services Research Foundation (CHSRF)
Cancer Care Nova Scotia
Cancer Care Ontario
Cancer Research Society
CURE Foundation
Fonds de la recherche en santé du Québec (FRSQ)
Heart & Stroke Foundation of Canada
Institut de recherche Robert-Sauvé en santé et en sécurité du travail (IRSST)
Michael Smith Foundation for Health Research
National Alliance for Autism Research
National Cancer Institute of Canada
Newton Foundation
REPAR (Réseau provincial de recherche en adaptation-réadaptation du Québec)
The Arthritis Society 

Appendix III: Meeting Evaluation
Comments from Participants

Participants liked the way the meeting was structured, with small group discussions followed by plenary sessions. They thought the facilitators were excellent and think that in future, it would be useful to have trained or designated facilitators assigned to each breakout group. Some would have liked to participate in more than one breakout group so they could discuss a variety of topics.  Some felt the focus was too biomedical/clinical and neglected the other pillars of health research.

Participants said that the meeting gave them the chance to share experiences, problems and solutions, one of the workshop objectives. They enjoyed the random table groupings and appreciated the casual, social atmosphere, but would have liked longer breaks to allow for more networking. Some felt the meeting should have followed the agenda and timetable more rigorously. Others wanted more emphasis on the practical elements of running a program and more opportunities to hear how different groups have solved common problems. They also would have liked some time set aside for people from training programs associated with specific institutes to meet with each other and an institute representative.

They liked hearing from Alan Bernstein. However, they wanted to hear more from CIHR regarding long-term plans (beyond the 6th year of programs), its expectations, and program management issues, such as stipend levels. They liked the fact that they will receive a meeting report but also want to hear how CIHR and its partners will implement some of the ideas presented at the meeting.