Sustainable Engineering Assessment Using Rubric-Based Analysis of Challenge Question Responses

1 INTRODUCTION Design education usually employs project-based learning, in which students learn to research, plan, design and reflect on the creation of technological projects [1], [2]. Student experience issues may influence students' design process, as well as their perception of professional knowledge development. Studies have pointed out students are ill-informed about design education when entering university [3]. In the field of design, although there have been student surveys to analyze design thinking skills and for assessing design literacy. However, these studies are primarily assessment tools, and do not investigate students’ attitudes or experiences of their design courses. While several studies focus on measuring students’ cognitive knowledge, assessments of affective variables, such as attitude, are far fewer in number. The objective of this study was to investigate student experiences, specifically the issues and challenges faced by students in three foundational design courses – focusing on introductory design, architectural design and engineering design – at the . We investigate issues such as how do students experience various instruction modes such as lectures and studio; and their experience of working individually or as teams in the three foundational design courses. 2 METHOD We conducted a survey in three foundational design courses, in both online and paper format. The survey comprised quantitative and qualitative components. Respondents consisted of 248 students from the introductory design course, 48 students from the introductory architectural design course and 71 students from the engineering design course. Respondents rated their perceptions on 23 items in two types of questions: Likert, semantic differential. Additionally, they provided their input in two open-ended questions. 3 RESULTS A principal component analysis conducted on the data revealed five factors that primarily contributed to students’ perception of foundational design courses: (1) the fun aspects of design, (2) the stressful aspects of design, (3) Course Pressures, (4) Student temperament issues, (5) Teamwork experience. Performance pressures majorly affected student experience in all three courses. The architectural design course was seen to be the most enjoyable for students, as fun scores were higher than the stress scores, and low course pressures. The engineering design course was observed to be very stressful for students, with low scores for fun. 4 SIGNIFICANCE Student perception of foundational design courses is important because it can be used to drive forward changes in curriculum and pedagogy. While this study uses data from one university, the issues reported here are generic and relevant for instructors developing design courses across different parts of the world. Research in student experience and perspectives is crucial to develop design education and propel it into the future. REFERENCES [1] Y. Doppelt, “Assessing creative thinking in design-based learning,” International Journal of Technology and Design Education, vol. 19, no. 1, pp. 55–65, 2009, [2] N. Lee, “Project methods as the vehicle for learning in undergraduate design education: a typology,” Design Studies, vol. 30, no. 5, pp. 541–560, Sep. 2009, [3] A. B. Gencosmanoglu, “Learning, teaching and administration in design education: DESIGNtrain Project: Training tools for developing design education,” in Procedia - Social and Behavioral Sciences, 2010.

This work reviews best practices in first-year engineering design courses at 40 universities across Canada and the United States. The authors reviewed the subject matter and instructional methods of these engineering design courses. University selection was based on prominence, level of engineering design content, and availability of data. The authors narrowed the scope of the study to seven Canadian programs and eight American programs for further investigation: University of British Columbia, University of Calgary, University of Manitoba, Queen’s University, University of Sherbrooke, University of Toronto, University of Western Ontario, University of Colorado, Franklin W. Olin Engineering College, Harvey Mudd College, Massachusetts Institute of Technology, Northwestern University, Rensselaer Polytechnic Institute, Stanford University, and Virginia Polytechnic Institute. The authors then identified six reoccurring themes in the methods of engineering design instruction: full-scale project, small-scale project, case study analysis, reverse engineering project, design tools and methods instruction, and integration. These themes are then discussed from the point of view of educators looking to develop first-year engineering design courses.

Assessment in capstone engineering design courses is vital to engineering education programs. The capstone design course is the climax of design education and often the context for much of the assessment done in engineering degree programs. Capstone design course instructors’ admittedly low confidence for assessing student performance in these courses poses a crucial obstacle to the assessment process. A key issue is a lack of clear outcomes definition for engineering design and sound, defensible assessments for these outcomes. This paper draws from findings in design literature and from engineering design education experience to construct a conceptual model of engineering design that guides development of associated design learning outcomes and assessment of student achievements in design. A conceptual model of design is vital to design education, because it is one of three essential cornerstones— model, observation, and interpretation— for constructing assessments. In the context of an engineering classroom, design performance produces two different and complementary types of outcomes: learner development and solution development. Further, design is: open-ended, iterative, creative, collaborative, goal-driven, process-intensive, productfocused, customer-oriented, value-added, and constrained by society. Learner and solution development usually progresses from a state of students’ fragmented understanding and ideas to a more mature state of integrated understanding and design solutions. The proposed conceptual model for engineering design identifies four areas of performance that describe design: (1) personal capacity, (2) team processes, (3) solution requirements, and (4) solution assets. The first two characterize learner development and the latter two characterize solution development. Personal capacity is the designer’s development of skills which support technically sound and responsible design. Team processes address a team’s behaviors and productivity in design activities. Solution requirements address the team’s understanding of stakeholder needs and concerns. Solution assets encompass the value added, practicality, and impact of the design solution. These four performance areas are essential to design. They identify performances that develop over time, contribute to the success of design outcomes, and stimulate growth in one another for integrated enhancement of design performance. They also provide the basis for a versatile, comprehensive definition of engineering design performance that can guide instruction and can focus assessments in capstone engineering design courses.

The purpose of this study is to consider how to use the e-Learning system in the engineering design courses at Kanazawa Institute of Technology (KIT). The courses consist of two levels: Engineering Design I (ED I) and Engineering Design II (ED II). The former is taken by all freshmen during the second quarter, and the latter is taken by all sophomores during the sixth quarter. KIT's calendar has three 10-week quarters per year. The courses require students to participate in a team project from the problem discovery stage to the concept design stage. As the first e-Learning trial in the engineering design course, we introduce the collaboration-communication function of the e-Learning system to our ED I class. In this paper, we describe the students' activities in each engineering design process in situations where students can use the e-Learning system.

The purpose of this study is to consider how to use an electronic learning (e-learning) system in engineering design. An e-learning trial in an engineering design course was introduced as an online self-study module in order to support learning outside of the classroom at the Kanazawa Institute of Technology. This paper contains a description of the e-learning system and the related student activities and teamwork.

technology [4] to a course management system of ED education as it has become a major technology which supports multimedia content publishing and sharing over the Internet. The authors proposed a concept of a new Web 2.0 course management system of ED education as the first step to complete the proposed system. Kanazawa Institute of Technology is a pioneering university in Japan that began its Engineering Design education in 1996. Our Engineering Design courses are characterized by project-based learning in groups. The current web-based information systems for Engineering Design education at KIT are composed of the following independent systems: a course website, an e-Learning system, an oral-presentation review system, and a report-submitting system. The systems are causing unnecessary confusion to both instructors and students, because it is composed of a variety of independent software and their human interfaces are not unified. 2. Learning activities of Engineering Design education A total of 1,700 engineering students from all fourteen engineering divisions must take ED courses. Fifty ED classes are offered in each year. A class consisting of approximately 34 students is divided into seven teams. Each team consists of 4 to 5 students. In order to maintain a high level of quality in the course management, a learning manual for students as well as a course management manual for instructors have been published and revised every year. The students must conduct two hours of in-class instructed learning as well as four hours of outside-class learning activities in teams per week for ten weeks. In our ED education, the Engineering Design Process (EDP) is defined as the process by which the following activities (stages) are performed. The authors analyzed students' learning activities in Engineering Design courses and then designed a new Web 2.0 course management system of Engineering Education in order to help students' learning. The paper shows the first stages of our research efforts toward the completion of a new Web 2.0 course management system of Engineering Education.

In the context of teaching design, engineers often have a strong preference for problem-based learning because the skills they are trying to teach are intrinsic to the solving of design problems. The proliferation of problem-based learning (PBL) in capstone and now cornerstone engineering design courses is well supported by industry and faculty and the trend has been towards seeing more PBL in engineering design courses. This paper explores the basic selection of engineering design problems and presents a fairly simple dilemma: the skills that are required to solve a problem are not necessarily the skills that are taught by the problem if the problem is truly open-ended. This paper presents the idea of using engineering problems that are carefully constructed simulacra of real-world problems with built in scaffolding to create PBL experiences for students that are educationally complete and meaningful. This paper presents examples from two courses developed at the University of Manitoba based on this approach, outcomes of and responses to the course layout, and ideas for how this model can be extended to other courses or programs.

In this study, we investigated the effects of students’ participation in a collaborative, project-based engineering design course on their domain knowledge, interests, and strategic processing. Participants were 70 college seniors working in teams on a design project of their choosing. Their declarative, procedural, and principled knowledge, along with their domain interest and their interest in select roles within that domain were tested at the outset of the semester and at its conclusions. Findings indicated that this course contributed to a rise in students’ declarative knowledge, but not their procedural or principled knowledge of engineering design. Further, there was no significant change in students’ personal interest in the domain over the semester, and their role interests were not associated with their demonstrated knowledge in the field at posttest. Implications for the perceived effectiveness of learning environments consisting of peer collaboration and relevant problems on students’ academic development are considered.

First Year Design courses are commonplace in many engineering curriculums. Although the focus of these courses typically revolves around introducing students to various multistep design processes as well as improving student skills in written and oral communication techniques; they can be limited in replicating the experience of working in a real world interdisciplinary design environment. In an industrial setting, design teams are comprised of members that have the complementary skills that are necessary to complete the relevant task. There are many tools, like CATME, available to replicate this process of designing student teams based on complementary skills. Prior studies have illustrated that, although assigned teams can improve the experience for some, it can also drastically diminish the student experience for others. This work focuses on the assignment of student design teams based on both complementary skills as well as shared interests. As part of the common first year engineering curriculum at Wentworth Institute of Technology in Boston, Ma, students participate on the Introduction to Engineering Design course. This work studies two years-worth of data on the impact of assigned groups in these courses, as they pertain to overall student experience. For this, various sections of the course adopted one of the following structures for group assignments: 1) student selected, 2) skills-based instructor assigned, 3) skills and student interest based instructor assigned. Written feedback and peer assessment, based on ABET Outcome D: Ability to function in multidisciplinary teams, were collected from the students. Our analysis focuses on the process of intelligently assigning student groups and the techniques that can improve the overall student experience. The authors hope to engage in a spirited discussion on the merits of assigned student design teams as well as propose an alternative approach for instructors planning to engage students in interdisciplinary project based courses.

The purpose of this work is to examine how project management is perceived in the engineering undergraduate curriculum at the University of Guelph across its four-year series of design courses, and to gather insight into the skills, elements, and level of mastery required. As a Canadian Engineering Accreditation Board graduate attribute, project management is taught in all engineering programs. The instructor perspective is explored through interviews with instructors that have taught engineering design courses. The instructor responses are analyzed for insights related to the effectiveness of the curriculum for the development of project management skills. This work is the first component a larger study that will examine and compare the findings with the views of students and alumni of the program. The results of these interviews aim to provide insight to engineering programs related to the level of mastery expected of graduating students, and future activities to explore for the purpose of closing gaps that might exist between graduate competencies and industry needs to better equip students with the skills of future leaders.

Problem-based learning techniques involving small collaborative teams of students are widely regarded as being effective in teaching engineering design courses. However, the heavy requirements upon the teaching staff to support and coordinate such teams typically limit these courses to small numbers of students. In this paper, we describe the design of a computer-based tool for supporting small collaborative teams with the ultimate goal of scaling these engineering design courses to larger numbers of students. The system, deployed on laptops with wireless networking, utilizes student self-evaluations and instrumented project management tools to provide feedback of individual and team performance to the students and teaching staff. We present an example of the usage and data collected from this system during a pilot test in an undergraduate computer science design course. Based on that experience, we discuss future plans to refine the system for early detection of problem teams and to instrument asynchronous and instant messaging tools to support social network analysis of team communication.

In 1990, the A. James Clark School of Engineering at the University of Maryland at College Park initiated a reevaluation of the introductory design course in order to increase design in the classroom and to recruit, retain, and graduate more women and underrepresented minorities. The resulting course, ENES 100, has been highly successful. A High School Teacher-in-Residence program was initiated in 1995. Sue Schwartz, a Physics teacher from Montgomery County's Paint Branch High School began a residency at the School of Engineering to learn and be a partner in the first year Introduction to Engineering Design course. Throughout the two semesters she actively participated in 4 sections of the course and interacted with all aspects of the freshman program. A major outcome of the interaction was a well defined, detailed, year-long high school curriculum in engineering design. That curriculum has had a pilot test throughout the 1996-97 academic year. The high school curriculum in engineering design and the results of the pilot study is the focus of the presentation.

Engineering design is a broad, yet essential topic for students to study during their academic tenure in the engineering field. Engineering design is the set of fundamental principles necessary to successfully solve a design problem. We propose a model for effective teaching of engineering design by developing a multifaceted engineering design course, whose goals are to introduce students to basic engineering design principles and professional skill methodologies, such as client interaction, teamwork, and presentation skills. We require students to make use of computer technologies taught in-class: Alias Maya for 3D design renderings and MathWorks Matlab for computational analysis of designs. Projects with community partners fulfill the need for real client interaction and robust design problems. Students gain hands-on experience from directly applying concepts taught in the course, while community partners benefit from the projects' research and proposed solution. Over 1000 first-year engineering students have participated in this course structure since its inception in Spring 2004. We have worked with over 50 community partners yielding more than 200 semester-length projects, some of which embark on a continuation over several semesters. Our experience complemented with extended evaluations after each semester shape the course each term to yield successful results.

To adequately prepare engineering students for their professional career, educational institutions offer projects in which students collaboratively solve engineering design problems. It is known from research these projects can lead to a variety of learning outcomes and student experiences. However, studies that provide insights in the influence of different features of an educational design are rare. In the current study we use Cultural Historical Activity Theory (CHAT) as analytical framework to understand how different elements of an educational design affect students’ experience. Additionally, we use the notion of contradictions to identify opportunities for structural course improvement. Focus groups were conducted with 12 Master students in Aerospace Engineering, that participated in a collaborative engineering design course. During the course, students applied Systems Engineering (SE) and Concurrent Engineering (CE) and worked in the Collaborative Design Laboratory (CDL), which is a state-of-the-art facility that holds a variety of industry relevant tools. It was found that students valued the guidance of their coach and experts, co-located collaboration and the freedom to structure their own process. However, they perceived challenges with regard to adoption of tools in the CDL, sharing their progress with their supervisor, coordination of collaborative efforts and scheduling issues. An analysis using CHAT revealed what contradictions caused these challenges. Finally, recommendations are given on how course structure can be structurally improved.

Texas Instruments (TI) currently offers a comprehensive curriculum to supplement their Robotics System Learning Kit MAX (TI-RSLK MAX). While designed for all college students, TI would like to provide a modification of the curriculum specifically for first-year engineering design courses. The goal of this project was to modify the current TI-RSLX MAX curriculum, making it easier to digest for first-year engineering students and more applicable to shorter duration projects (∼8 weeks), while still being information dense. To better understand the user experience of the current TI-RSLK MAX curriculum and determine what users would want from a modified curriculum, user interviews were conducted and analyzed using inductive qualitative analysis techniques. Results showed that while the current TI robotics project has many positives, first-year students primarily struggled with the coding aspect of the project. These results directed and informed the prototype development of a modified curriculum module focused on teaching students code. Low- and medium-fidelity prototypes were developed and tested with users. Data from the usability study were analyzed using descriptive analysis techniques and results indicate that the medium-fidelity prototype helped students learn robotics coding. This research has implications for a high-fidelity modified TI-RSLX MAX curriculum and robotics education for first-year engineering design students.