DEVELOPING A FRAMEWORK TO EVALUATE INDIVIDUAL LEARNING IN ENGINEERING DESIGN PROBLEMS – PART 1: TEACHING THROUGH SIMULACRA OF REAL-WORLD PROBLEMS

1 INTRODUCTION Design education usually employs project-based learning, in which students learn to research, plan, design and reflect on the creation of technological projects [1], [2]. Student experience issues may influence students' design process, as well as their perception of professional knowledge development. Studies have pointed out students are ill-informed about design education when entering university [3]. In the field of design, although there have been student surveys to analyze design thinking skills and for assessing design literacy. However, these studies are primarily assessment tools, and do not investigate students’ attitudes or experiences of their design courses. While several studies focus on measuring students’ cognitive knowledge, assessments of affective variables, such as attitude, are far fewer in number. The objective of this study was to investigate student experiences, specifically the issues and challenges faced by students in three foundational design courses – focusing on introductory design, architectural design and engineering design – at the . We investigate issues such as how do students experience various instruction modes such as lectures and studio; and their experience of working individually or as teams in the three foundational design courses. 2 METHOD We conducted a survey in three foundational design courses, in both online and paper format. The survey comprised quantitative and qualitative components. Respondents consisted of 248 students from the introductory design course, 48 students from the introductory architectural design course and 71 students from the engineering design course. Respondents rated their perceptions on 23 items in two types of questions: Likert, semantic differential. Additionally, they provided their input in two open-ended questions. 3 RESULTS A principal component analysis conducted on the data revealed five factors that primarily contributed to students’ perception of foundational design courses: (1) the fun aspects of design, (2) the stressful aspects of design, (3) Course Pressures, (4) Student temperament issues, (5) Teamwork experience. Performance pressures majorly affected student experience in all three courses. The architectural design course was seen to be the most enjoyable for students, as fun scores were higher than the stress scores, and low course pressures. The engineering design course was observed to be very stressful for students, with low scores for fun. 4 SIGNIFICANCE Student perception of foundational design courses is important because it can be used to drive forward changes in curriculum and pedagogy. While this study uses data from one university, the issues reported here are generic and relevant for instructors developing design courses across different parts of the world. Research in student experience and perspectives is crucial to develop design education and propel it into the future. REFERENCES [1] Y. Doppelt, “Assessing creative thinking in design-based learning,” International Journal of Technology and Design Education, vol. 19, no. 1, pp. 55–65, 2009, [2] N. Lee, “Project methods as the vehicle for learning in undergraduate design education: a typology,” Design Studies, vol. 30, no. 5, pp. 541–560, Sep. 2009, [3] A. B. Gencosmanoglu, “Learning, teaching and administration in design education: DESIGNtrain Project: Training tools for developing design education,” in Procedia - Social and Behavioral Sciences, 2010.

This work reviews best practices in first-year engineering design courses at 40 universities across Canada and the United States. The authors reviewed the subject matter and instructional methods of these engineering design courses. University selection was based on prominence, level of engineering design content, and availability of data. The authors narrowed the scope of the study to seven Canadian programs and eight American programs for further investigation: University of British Columbia, University of Calgary, University of Manitoba, Queen’s University, University of Sherbrooke, University of Toronto, University of Western Ontario, University of Colorado, Franklin W. Olin Engineering College, Harvey Mudd College, Massachusetts Institute of Technology, Northwestern University, Rensselaer Polytechnic Institute, Stanford University, and Virginia Polytechnic Institute. The authors then identified six reoccurring themes in the methods of engineering design instruction: full-scale project, small-scale project, case study analysis, reverse engineering project, design tools and methods instruction, and integration. These themes are then discussed from the point of view of educators looking to develop first-year engineering design courses.

Assessment in capstone engineering design courses is vital to engineering education programs. The capstone design course is the climax of design education and often the context for much of the assessment done in engineering degree programs. Capstone design course instructors’ admittedly low confidence for assessing student performance in these courses poses a crucial obstacle to the assessment process. A key issue is a lack of clear outcomes definition for engineering design and sound, defensible assessments for these outcomes. This paper draws from findings in design literature and from engineering design education experience to construct a conceptual model of engineering design that guides development of associated design learning outcomes and assessment of student achievements in design. A conceptual model of design is vital to design education, because it is one of three essential cornerstones— model, observation, and interpretation— for constructing assessments. In the context of an engineering classroom, design performance produces two different and complementary types of outcomes: learner development and solution development. Further, design is: open-ended, iterative, creative, collaborative, goal-driven, process-intensive, productfocused, customer-oriented, value-added, and constrained by society. Learner and solution development usually progresses from a state of students’ fragmented understanding and ideas to a more mature state of integrated understanding and design solutions. The proposed conceptual model for engineering design identifies four areas of performance that describe design: (1) personal capacity, (2) team processes, (3) solution requirements, and (4) solution assets. The first two characterize learner development and the latter two characterize solution development. Personal capacity is the designer’s development of skills which support technically sound and responsible design. Team processes address a team’s behaviors and productivity in design activities. Solution requirements address the team’s understanding of stakeholder needs and concerns. Solution assets encompass the value added, practicality, and impact of the design solution. These four performance areas are essential to design. They identify performances that develop over time, contribute to the success of design outcomes, and stimulate growth in one another for integrated enhancement of design performance. They also provide the basis for a versatile, comprehensive definition of engineering design performance that can guide instruction and can focus assessments in capstone engineering design courses.

The purpose of this study is to consider how to use the e-Learning system in the engineering design courses at Kanazawa Institute of Technology (KIT). The courses consist of two levels: Engineering Design I (ED I) and Engineering Design II (ED II). The former is taken by all freshmen during the second quarter, and the latter is taken by all sophomores during the sixth quarter. KIT's calendar has three 10-week quarters per year. The courses require students to participate in a team project from the problem discovery stage to the concept design stage. As the first e-Learning trial in the engineering design course, we introduce the collaboration-communication function of the e-Learning system to our ED I class. In this paper, we describe the students' activities in each engineering design process in situations where students can use the e-Learning system.

To adequately prepare engineering students for their professional career, educational institutions offer projects in which students collaboratively solve engineering design problems. It is known from research these projects can lead to a variety of learning outcomes and student experiences. However, studies that provide insights in the influence of different features of an educational design are rare. In the current study we use Cultural Historical Activity Theory (CHAT) as analytical framework to understand how different elements of an educational design affect students’ experience. Additionally, we use the notion of contradictions to identify opportunities for structural course improvement. Focus groups were conducted with 12 Master students in Aerospace Engineering, that participated in a collaborative engineering design course. During the course, students applied Systems Engineering (SE) and Concurrent Engineering (CE) and worked in the Collaborative Design Laboratory (CDL), which is a state-of-the-art facility that holds a variety of industry relevant tools. It was found that students valued the guidance of their coach and experts, co-located collaboration and the freedom to structure their own process. However, they perceived challenges with regard to adoption of tools in the CDL, sharing their progress with their supervisor, coordination of collaborative efforts and scheduling issues. An analysis using CHAT revealed what contradictions caused these challenges. Finally, recommendations are given on how course structure can be structurally improved.

First Year Design courses are commonplace in many engineering curriculums. Although the focus of these courses typically revolves around introducing students to various multistep design processes as well as improving student skills in written and oral communication techniques; they can be limited in replicating the experience of working in a real world interdisciplinary design environment. In an industrial setting, design teams are comprised of members that have the complementary skills that are necessary to complete the relevant task. There are many tools, like CATME, available to replicate this process of designing student teams based on complementary skills. Prior studies have illustrated that, although assigned teams can improve the experience for some, it can also drastically diminish the student experience for others. This work focuses on the assignment of student design teams based on both complementary skills as well as shared interests. As part of the common first year engineering curriculum at Wentworth Institute of Technology in Boston, Ma, students participate on the Introduction to Engineering Design course. This work studies two years-worth of data on the impact of assigned groups in these courses, as they pertain to overall student experience. For this, various sections of the course adopted one of the following structures for group assignments: 1) student selected, 2) skills-based instructor assigned, 3) skills and student interest based instructor assigned. Written feedback and peer assessment, based on ABET Outcome D: Ability to function in multidisciplinary teams, were collected from the students. Our analysis focuses on the process of intelligently assigning student groups and the techniques that can improve the overall student experience. The authors hope to engage in a spirited discussion on the merits of assigned student design teams as well as propose an alternative approach for instructors planning to engage students in interdisciplinary project based courses.

Problem-based learning techniques involving small collaborative teams of students are widely regarded as being effective in teaching engineering design courses. However, the heavy requirements upon the teaching staff to support and coordinate such teams typically limit these courses to small numbers of students. In this paper, we describe the design of a computer-based tool for supporting small collaborative teams with the ultimate goal of scaling these engineering design courses to larger numbers of students. The system, deployed on laptops with wireless networking, utilizes student self-evaluations and instrumented project management tools to provide feedback of individual and team performance to the students and teaching staff. We present an example of the usage and data collected from this system during a pilot test in an undergraduate computer science design course. Based on that experience, we discuss future plans to refine the system for early detection of problem teams and to instrument asynchronous and instant messaging tools to support social network analysis of team communication.

Engineering design is a broad, yet essential topic for students to study during their academic tenure in the engineering field. Engineering design is the set of fundamental principles necessary to successfully solve a design problem. We propose a model for effective teaching of engineering design by developing a multifaceted engineering design course, whose goals are to introduce students to basic engineering design principles and professional skill methodologies, such as client interaction, teamwork, and presentation skills. We require students to make use of computer technologies taught in-class: Alias Maya for 3D design renderings and MathWorks Matlab for computational analysis of designs. Projects with community partners fulfill the need for real client interaction and robust design problems. Students gain hands-on experience from directly applying concepts taught in the course, while community partners benefit from the projects' research and proposed solution. Over 1000 first-year engineering students have participated in this course structure since its inception in Spring 2004. We have worked with over 50 community partners yielding more than 200 semester-length projects, some of which embark on a continuation over several semesters. Our experience complemented with extended evaluations after each semester shape the course each term to yield successful results.

The purpose of this study is to consider how to use an electronic learning (e-learning) system in engineering design. An e-learning trial in an engineering design course was introduced as an online self-study module in order to support learning outside of the classroom at the Kanazawa Institute of Technology. This paper contains a description of the e-learning system and the related student activities and teamwork.

technology [4] to a course management system of ED education as it has become a major technology which supports multimedia content publishing and sharing over the Internet. The authors proposed a concept of a new Web 2.0 course management system of ED education as the first step to complete the proposed system. Kanazawa Institute of Technology is a pioneering university in Japan that began its Engineering Design education in 1996. Our Engineering Design courses are characterized by project-based learning in groups. The current web-based information systems for Engineering Design education at KIT are composed of the following independent systems: a course website, an e-Learning system, an oral-presentation review system, and a report-submitting system. The systems are causing unnecessary confusion to both instructors and students, because it is composed of a variety of independent software and their human interfaces are not unified. 2. Learning activities of Engineering Design education A total of 1,700 engineering students from all fourteen engineering divisions must take ED courses. Fifty ED classes are offered in each year. A class consisting of approximately 34 students is divided into seven teams. Each team consists of 4 to 5 students. In order to maintain a high level of quality in the course management, a learning manual for students as well as a course management manual for instructors have been published and revised every year. The students must conduct two hours of in-class instructed learning as well as four hours of outside-class learning activities in teams per week for ten weeks. In our ED education, the Engineering Design Process (EDP) is defined as the process by which the following activities (stages) are performed. The authors analyzed students' learning activities in Engineering Design courses and then designed a new Web 2.0 course management system of Engineering Education in order to help students' learning. The paper shows the first stages of our research efforts toward the completion of a new Web 2.0 course management system of Engineering Education.

This paper draws on research in both technical communication and education to describe a problem-based learning approach to engineering design courses that more authentically engages students in the practice of communicating with specific audiences for specific purposes. By developing a learner-centered classroom in which faculty serve as mentor-managers rather than repositories of knowledge, instructors can develop communication tasks that more closely reflect the way texts function in the workplace. In doing so, they can better prepare students for role that texts play in professional practice and enable them to make decisions accordingly. The author includes results from pilot sections of capstone engineering design courses to describe the implementation and evaluate the effectiveness of the approach.

– Inconsistent transfer policies, lack of articulated syllabi, and subsequent differences on the delivery of comparable courses are a few examples of the obstacles that Ontario students face when trying to change programs. This study sought to develop a framework to support transfer between engineering and engineering technology programs in Ontario using explicit and implicit course outcomes to help develop and define new pathways. Primarily focusing on program expectations in introductory design courses this study compared the content and context of design projects in different institutions and programs across the province. The contextual framework, namely the “Outcome Comparison Framework for Engineering and Technology”, synthesized relevant elements from existing frameworks which can collectively identify the differences in the context of learning in engineering and technology disciplines. This framework looks into disciplinarity, use of tools and design thinking required to successfully finish a design project. We collected design projects from 5 technology programs and 4 engineering programs across the province and coded them based on the content targeted by the project description and the three dimensions of the framework. Content analysis for design courses showed an overall low percentage of alignment between the learning outcomes and the project descriptions across the board. It was found that engineering design courses were more focused on principles of engineering design (problem definition, stakeholder needs, idea generation, decision making) and development of professional characteristics (workplace communication skills, ethics, etc.); but technology design courses, focused on the use of more “hands-on”' skills (building/implementation, troubleshooting, etc.)

In this study, we investigated the effects of students’ participation in a collaborative, project-based engineering design course on their domain knowledge, interests, and strategic processing. Participants were 70 college seniors working in teams on a design project of their choosing. Their declarative, procedural, and principled knowledge, along with their domain interest and their interest in select roles within that domain were tested at the outset of the semester and at its conclusions. Findings indicated that this course contributed to a rise in students’ declarative knowledge, but not their procedural or principled knowledge of engineering design. Further, there was no significant change in students’ personal interest in the domain over the semester, and their role interests were not associated with their demonstrated knowledge in the field at posttest. Implications for the perceived effectiveness of learning environments consisting of peer collaboration and relevant problems on students’ academic development are considered.

A method to evaluate students’ application of sustainability principles into engineering design and problem solving was developed. Students were presented with two challenge questions that each posed a scenario. Students responded to one question and their answers were scored using a rubric that combined both analytic and holistic scoring criteria to account for sustainability principles. The rubric was iteratively developed and met standard measures of reliability and validity. Based on responses from primarily civil and environmental engineering students at three institutions, it was found that male students, seniors, and participants in many diverse learning activities (extracurricular service, internships, undergraduate research, and engineering design courses) achieved higher rubric scores. Scores on the challenge questions did not correlate to students’ interests in sustainable engineering, which were measured using a Likert-based survey. A challenge-question approach to formative assessment may enable instructors to adapt curricula to foster student learning of implicit sustainability issues in engineering. The rubric may be used with a diversity of content-focused challenge questions as a tool for assessing students’ holistic, sustainability-focused approaches to engineering.

The purpose of this work is to examine how project management is perceived in the engineering undergraduate curriculum at the University of Guelph across its four-year series of design courses, and to gather insight into the skills, elements, and level of mastery required. As a Canadian Engineering Accreditation Board graduate attribute, project management is taught in all engineering programs. The instructor perspective is explored through interviews with instructors that have taught engineering design courses. The instructor responses are analyzed for insights related to the effectiveness of the curriculum for the development of project management skills. This work is the first component a larger study that will examine and compare the findings with the views of students and alumni of the program. The results of these interviews aim to provide insight to engineering programs related to the level of mastery expected of graduating students, and future activities to explore for the purpose of closing gaps that might exist between graduate competencies and industry needs to better equip students with the skills of future leaders.