Teaching and learning computer science sorting algorithms with mobile devices: A case study
Teaching and LearningComputer Science SortingAlgorithms With MobileDevices: A Case StudyIVICA BOTICKI,1 ANTE BARISIC,1 SERGIO MARTIN,2 NEVEN DRLJEVIC11Faculty of Electrical Engineering and Computing, Department of Applied Computing, University of Zagreb, Zagreb,Croatia2Electrical and Computer Engineering Department, UNED (Spanish University for Distance Education), Madrid, SpainReceived 3 December 2011; accepted 20 February 2012ABSTRACT: In this article, we present a semester-long study on using smartphone devices in computingengineering education. We developed Sortko, an Android-based smartphone application for learning sortingalgorithms, an important undergraduate computer science topic. The application consists of four main compo-nentsthe module for interactive sorting, the scaffolding module, the motivational module, and the graphicaluser interface module, each with a distinct role of helping students in learning sorting algorithms. Our re-search methodology included data collection with administered two surveys, collected exam results andrecorded application usage data. Analysis of the collected data shows our approach is an effective way oflearning sorting algorithms. 2012 Wiley Periodicals, Inc. Comput Appl Eng Educ; View this article online atwileyonlinelibrary.com/journal/cae; DOI 10.1002/cae.21561Keywords: mobile learning; engineering education; sorting algorithmsINTRODUCTIONThis study examines educational benets of smartphone devicesin higher education. In our approach students use Android devices as a support in the process of learning sorting algo-rithms. Sorting, or putting series of elements in appropriate or-der, is an important computer science topic and presents an oldand well-dened computer science problem. Mastering sortingalgorithms is especially important for future software engineersregardless of the specic domain they work in.The growing popularity of smartphone devices can beclearly observed in everyday student life. Although they aremostly used for fun and informal ways of learning (e.g., accessto Wikipedia, YouTube, etc.), scientic studies have shownthe educational value of these devices. Based on an informalobservation of our students interest in learning with smart-phone devices and by taking into account behaviorism and so-cialconstructivist theory of learning, we are suggesting adesign of a smartphone application for learning sorting algo-rithms which, amongst others, includes a scaffolding moduledesigned to assist students while performing sorting and a moti-vational module to additionally motivate students in using theapplication.Normally, students learn sorting by using a paper and pen-cil while simultaneously watching sorting algorithm programcode. This tiring and ineffective procedure often results in dif-ferent kinds of errors such as students learning their own ver-sion of a sorting algorithm which does not match the real one(the sequence of steps while sorting elements is not adequate).We believe that technology can restructure this troublesome andineffective process of learning thereby improving studentsknowledge of sorting algorithms. To verify our assumptions weare exploring these research questions: The students who used the application achieve betteracademic results than the students who did not use theapplication. The scaffolding module is able to assist students in theprocess of sorting. The motivational module motivates students on furtherapplication use.These research questions will be explored by analyzingresults of standard exams, usage logs, and specially designedCorrespondence to I. Boticki (email@example.com). 2012 Wiley Periodicals, Inc.1questionnaires querying our students perception on usage andusefulness of the Sortko application and the approach ingeneral.Learning and Playing With Mobile DevicesAn early research conducted in England as early as in 2005 hadshown that more than 95% of students possess at least one mo-bile device , which opens up a space for their use in theprocess of educating young people. As for the effectiveness ofthe approach, another study presents a learning trail in whichstudents were equipped with 100 Palm Pads stating that 90% ofteachers had a positive attitude regarding the impact of thistype of technology on students and on their apprehension oflearning content [3,4].Apart from mobile learning [5,6], game-based learningexamines the process of studying which is catalyzed and stimu-lated with the elements that are considered to be the standardparts of games (i.e., awards, collecting points, reaching higherlevels, and achieving high scores in computer games). Thesemotivate players to be involved in the game more thus enhanc-ing and accelerating their learning process through repeatedgame play .The crucial part of such approaches is the united goal ofthe game and of the learning process [8,9]. Once that isachieved, learning becomes entertainment and students get mo-tivated to learn at home and to take true interest in the coursecontents making the educational process more efcient andself-regulated . What is more, by employing new pedago-gies which integrate learning with game play, teaching becomesmore fun as well.A Paper and Pencil Approach to Learning SortingAlgorithmsThe topic of learning sorting algorithms is a part of the Algo-rithms and Data Structures course at the Faculty of ElectricalEngineering and Computing, University of Zagreb, Croatia andhas so far been taught in a paper-and-pencil fashion. Typically,various types of sorting algorithms (such as the Bubble sortalgorithm which bubbles out the greatest number in eachalgorithm pass) [11,12] are visualized by drawing a series ofalgorithm passes, where each pass is drawn as an array of num-bers on the blackboard or a sheet of paper. After each algorithmpass, a new sequence of numbers is written down again(Fig. 1). As a consequence, during exams, students knowledgeof sorting algorithms is tested in the same way.This repetitive and cumbersome way of learning oftenresults in errors (i.e., a wrong sequence of algorithm steps islearned) and distracts students from actually understanding thealgorithm big picture. Another problem that arises from thistype of learning is that there is no easy way a student can checkthe accuracy of her proposed sorting algorithm passes. What ismore, students sometimes get a correct (sorted) sequence evenwhen a wrong set of algorithm steps is applied. For example,while applying Quick sort algorithm and identifying the pivotalelement, a student can decide not to follow the step-to-step ap-proach of the algorithm and just position all smaller numberson the left-hand side of the pivotal element and all greater num-bers on the right-hand side of the pivotal element. Althoughconceptually correct and with a correct nal solution, this wayof sorting does not indicate whether the student actually knowsthe precise sequence of steps of the Quick sort algorithm andhence is graded with lower marks.The Use of Animations in Learning Sorting AlgorithmsThe use of animation materials as learning resources has beenadvocated by many researchers  and there are a number ofstudies dealing with overcoming the typical difculties in learn-ing sorting algorithms with the use of animations [14,15]. Thus,many authors  have provided animated resources totheir students in order to enhance the concepts acquisition.Sutinen  introduced a Web-based tool, Jeliot, providingsimple animations helping students in learning and understand-ing algorithms and data structures. Kordaki  performed apilot where secondary school students used a web-based appli-cation helping them in understanding different sorting algo-rithms during in-class learning.An algorithm animation helps students in the visualizationof a specic sorting algorithm but does not allow them to tryout sorting on their own . For example, www.sorting-algo-rithms.com web page  with a parallel animation of severalsorting algorithms on the same unsorted number sequence.With this simulation several sorting algorithm behaviors can beobserved on a randomly generated, almost sorted, reversed or asequence with repeated numbers. Figure 2 shows the Sort Ani-mation web page which, in addition to the sorting algorithmoperation animation, allows a parallel view of the programmingcode being executed during a specic animation step. Theapplication also shows temporary algorithm variables and pro-vides users with a possibility of adjusting the sorting sequencelength.In general, animations help viewers track patterns and ob-serve relationships, which is claimed to improve learning .However, some researchers challenge this stating that the use ofPC-based animations for algorithm visualization does not nec-essarily lead to better concepts acquisition than with static dia-grams, especially in the case of students needing to makeFigure 1 Traditional visual representation of the Quick sort sortingalgorithm operation while sorting an array of numbers. [Colorgure can be viewed in the online issue, which is available atwileyonlinelibrary.com.]2 BOTICKI ET AL.predictions of the coming animation sequence [13,15,25,26]and even suggest that animations distract students. Building onthe rich amount of knowledge in learning with the help of ani-mations we propose a design in which instructional agencyshifts toward the students side. By allowing students to explorethe rich space of both incorrect and correct solutions while sort-ing, we aim to utilize technology capabilities to provide themwith scaffolds in an experience-driven learning approach.MATERIALS AND METHODSSortko System in UseThe smartphone mobile learning application Sortko, designedas a part of this study, provides students with a graphical userinterface with an option of choosing a sorting algorithm (e.g.,bubble sort, shell sort, quick sort, insertion sort, etc.) and gener-ates a random sequence of numbers to be sorted by applying asequence of specic interaction gestures. During the sortingprocess, the system informs students about their success in per-forming so-called switches (element exchanges) and pro-vides them with textual help messages facilitating furtherprogress. For each game a student gets rewarded with pointsthat are then summarized on a joint public list.The graphical user interface consists of three main views:initial view, sorting view, and the result view, all made in hori-zontal mode to maximize display space and usage ergonomics(left-hand side of Fig. 3a).The sorting view contains a sequence of randomly gener-ated numbers (Fig. 3a, callout 1). While students sort by apply-ing gestures (Table 1), the system displays contextual textualhelp messages (Fig. 3a, callout 3). After each step (pass) of asorting process, the textual help message changes thereforedirecting students toward the next required action. The current-ly used algorithm is shown to users as well (Fig. 3a, callout 4)indirectly determining the number of points a student will beassigned according to the predened formula (details of the mo-tivational module are described in the Motivational ModuleSection).The Sortko application contains an auxiliary visual ele-ment positioned separately from the sorting array (Fig. 3a,callout 2). It is used to hold a variable with a temporarilyelement value stored until certain algorithm condition getsfullled.After sorting has completed, the results are sent to theserver side and the complete ranking list is shown to the sorter(Fig. 3b).Sortko Application ModelSortko System Architecture. Sortko application is built for An-droid operating system which includes a set of programminglibraries corresponding to the elementary Java programminglanguage library . Every Android application runs in its ownprocess, with its own instance of the Dalvik virtual machine(VM). Dalvik  is a purpose-built VM designed specicallyfor Android taking into account inherent mobile device limita-tions such as short battery life and processing power. Sortkoapplication is created using Eclipse integrated developmenttools (IDE) equipped with the Android Development Toolsplugin (ADT plugin) which allows for an easy setup of Androidprojects, application UIs and for the Android Framework APIcomponent reuse .The Sortko system is based on the clientserver architec-ture model and is divided into the following components: dataaccess layer, the domain model, web component, service com-ponent, and the smartphone client application (Fig. 4). Theserver side of the system consists of a database, data accesslayer, web services, and a web site. The web part resides on thedata access layer and can be accessed via desktop computersand mobile devices.The client part of the system is developed as an Androidsmartphone application and consists of the following modules(Fig. 5): The Interactive sorting module; The Scaffolding module providing help messages;Figure 2 Sort Animation web page with a parallel view of a sortinganimation and the appropriate algorithm execution (http://www.cs.oswego.edu/mohammad/classes/csc241/samples/sort/Sort2-E.html). [Color gurecan be viewed in the online issue, which is available at wileyonlinelibrary.com.]Figure 3 Graphical user interface of the Sortko Android smartphone application: (a) sorting view and (b) resultsview. [Color gure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]SORTING WITH MOBILE DEVICES 3 The Motivational module for awarding students; The User interface module.Graphical User Interface Module. The Interactive sortingmodule veries whether sorting steps provided by a student arevalid for a given sorting algorithm. The Scaffolding modulehelps students in solving typical dilemmas in the sorting pro-cess and displays help messages after an error in sorting hasbeen identied. It directs users toward choosing the right sort-ing solution by illustrating the logic behind a sorting problem,which is crucial in understanding sorting algorithms. The Moti-vational module tracks the successfulness of sorting accordingto different parameters (speed, accuracy, etc.) and awards pointsto users accordingly. The summary points and ranks are pre-sented in form of a web page accessible to all students.The client side application architecture follows the Model-View-ViewModel pattern . It consists of a view, which col-lects user input and forwards it to the viewmodel and whichactively pulls the data from the viewmodel by using the databinding technique. This pattern facilitates desirable software de-velopment techniques such as the separation of concerns(decoupling), unit testing, and test driven development. View-model encompasses all the benets of the patterns such as Mod-el-View-Controller and in the same time has an additionalbenet: the simplication coming from the declarative bindingtransferring data bidirectionally between viewmodel and theview.Interactive Sorting Module. The Interactive sorting modulecontains specialized implementations of sorting algorithms usedto verify whether a user has chosen the accurate algorithmsteps. The base class inherited by all specialized algorithmimplementations is the abstract Algorithm.java class. Concreteimplementations of algorithms are (1) BubbleSort.java, (2)InsertionSort.java, (3) QuickSort.java, (4) SelectionSort.java,and (5) ShellSort.java (Fig. 6).Scaffolding Module. The Scaffolding module helps students insorting by giving them contextual textual help messages withthe instructions on how to continue or on how to correct anerror. It is linked with the Interactive sorting module and pre-serves the current state of the sorting algorithm currently in use(Fig. 7).The contextual text help messages are implementedaccording to each algorithms specics and capture current al-gorithm position. For example, textual help messages for theBubble sort algorithm are provided by the BubbleSortPosition.-java class.Motivational Module. The Motivational module assigns pointsto students which are then sent to the server-side and there getsummarized. The amount of points a student is awardeddepends on the accuracy and the speed of her actions and thetype of algorithm used. At a microlevel (one user action or oneelement exchange), points for a successful user action areawarded according to the following formula:User Action Points 1; 000 Alg Weight 10; 000 12 Alg Weight=10Switch DurationThe formula is designed specically for this project in or-der to reward the use of more difcult algorithms. It consists ofan exponential decreasing function starting with the maximumnumber of points, which decreases over time. For moreTable 1 Typical Sorting GesturesSorting gesture Sorting gesture descriptionSorting gesturealgorithm coverage NoteElement switch Two sequence elements switch their positions Bubble sort, Selection sort Action sequence not importantElement switch(with itself)Element gets switched with itself, no visiblesequence change afterwardsQuick sort, Selection sortElement copy One element value is copied into another element Shell sort, Insertion sort Action sequence is importantFigure 4 Sortko system architecture. [Color gure can be viewed inthe online issue, which is available at wileyonlinelibrary.com.] Figure 5 Client side of the Sortko system.4 BOTICKI ET AL.complex algorithms, the function is declining more slowly butstarts from a much larger number of points, therefore leavingmore time for choosing an algorithm step to students. In caseof simple sorting algorithms, the function is adjusted so that itrewards speed and accuracy, and has a smaller contribution tothe overall amount of collected points. Such a reward systemshould direct users toward using more complex algorithms.The duration of one element switch (Switch_Duration) isthe time from the last successful element switch till the currentelement switch. The formula constant 1,000 represents minimalpoints a student can get per one action (one element exchange),while the constant 10,000 represents the maximum number ofpoints a user can get per one action. Constants 2 and 10 areused to model exponential function used to reduce the numberof points awarded per user action as the exchange durationincreases. As a consequence, a student gets a minimum of1,000 points for a successful action no matter how long it tookto complete, and a maximum of Alg_Weight 10,000 pointsfor a successful action with the shortest possible element ex-change duration (if her action happened really fast). The algo-rithm weight (Alg_Weight) is a constant specic to each sortingalgorithm used to reward students with more points when sort-ing with Quick sort and Shell sort algorithms. For an unsuc-cessful action users get penalized with a xed amount of 1,000points. The nal game points are calculated by summing up allpoints for successful actions and by deducting all points comingfrom the unsuccessful actions, therefore forming total singlegame points.Figure 8 shows two graphs with the visualization of theUser_Action_Points formula for the two distinct algorithmweight (Alg_Weight) constants 1 (Bubble Sort, Insertion Sortand Selection Sort) and 5 (Quick Sort and Shell Sort algorithms).The graphs show that the function gives more points to themore demanding algorithms. For example, for a switch intervalof 5 s, a student is awarded with around 1,300 points in the rstcase (Alg_Weight 1) and with around 7,500 points in thesecond case (Alg_Weight 5).RESEARCH DESIGN, DATA COLLECTION,AND RESULTSSortko was used in the summer semester of the academic year2010/2011 of the Algorithms and Data Structures coursetaught as a part of the undergraduate study at the Faculty ofElectrical Engineering, University of Zagreb. Prior to thestudents being formally introduced with the topic of learningsorting algorithms, the Sortko system was presented and theinitial survey was administered.During the classes covering sorting algorithms studentswere able to download and install the Sortko application ontotheir Android smartphones from the Sortko website .They were enticed to do so by the teacher, but there was nocurriculum integration of the Sortko activities, neither werethere additional merits for students. Sortko was deployed as anadd-on and its value is examined from that perspectivethroughout this section of the paper. Although, initially, theapplication was available for download as an Android-specicapk le, the new Sortko version can be found on the AndroidMarket and downloaded as any other Android smartphoneapplication .After the topic of sorting algorithm was completed, thestudents took their midterm exams. In addition to the midtermexam results, the log information on students activity in theSortko system was collected and analyzed. As the nal datacollection mechanism in this study, another survey was admin-istered in order to gather the overall students experiences.The Initial SurveyPrior to the Sortko application release, the initial anonymoussurvey was administered. One hundred twenty-two out of 596students enrolled in the course took part in this survey. It con-sisted of 20 questions mainly targeting students afnities to-ward the use of technology in learning, availability of Android-based smartphones, etc. (Tables 2 and 3).Figure 6 Class diagram showing the main classes of the Interactive sorting module. [Color gure can be viewed inthe online issue, which is available at wileyonlinelibrary.com.]Figure 7 Class diagram showing the main classes of the Scaffolding module. [Color gure can be viewed in theonline issue, which is available at wileyonlinelibrary.com.]SORTING WITH MOBILE DEVICES 5The survey results show that sorting algorithms present anew topic to the students. Although the majority has heardabout sorting, as much as 69% claim they do not possess opera-tional knowledge of sorting. This nding eliminates the tertiumquid effect of the prior knowledge of sorting that might inter-fere with our study results. Additionally, we were interested innding out whether our students were keen on accepting thenew way of studying with smartphones. As the survey shows,they feel they learn best when they do it home alone studyingcourse literature (67%) with only 28% applying some form ofcollaborative learning. This does not come as a surprise consid-ering half of the students are not satised with the formal lec-tures as they are currently done by the teachers in classrooms.This nding not only opens up a new space for curriculum in-novation, but also cautions us as our students are generally notsatised with the current learning and teaching system.As a digital teaching tool, Sortko differs from the toolsstudents are used to in their prior education. This does not pres-ent a problem to our students who believe that the technologycan be used for learning (93%) and that it can enhance educa-tion (96%). They also believe sorting algorithms can be learnedby using computers (90%) and as much as 85% believe that itcan be done better than without computers. As far as smart-phones are concerned, the students are unsure whether they canhelp in learning. However, the students who already own asmartphone gave more optimistic responses (AVG 3.41)compared to the students who do not have a smartphone device(AVG 2.86), with the mean difference of 0.55 beingsignicant t(120) 2.08, P < .05 two-tailed, r .18, repre-senting a small-sized effect. Similar results were acquired forthe general questions on use of technology in education.In total 39% of the surveyed students have their ownsmartphone device and 62% of smartphone users own anAndroid-based device, with version 2.2 being the most frequent(52%).Sortko Usage and Midterm Results AnalysisAfter each successful sequence sorting, Sortko sends studentidentication, sorting algorithms applied, and the score awardedto the student to the server-side storage. That data is used toanalyze what kind of sorting algorithms are most frequent andwhether usage correlates with the standard exam results. Thelog data shows the most frequently used sorting algorithm wasBubble sort (31% of the total sequence sorting done), followedby Selection Sort (27%), Quick Sort (17%), Insertion Sort(15%), and the least used Shell sort (10%). We explain thisdifference with the two main reasons: Bubble sort is the sim-plest and usually the rst algorithm taught.The course Algorithms and data structures includes twomidterm exams and a nal exam. Sorting algorithms are taughtas the middle part of the course and are assessed with an as-signment in the second midterm exam, such as this:A sequence of numbers 8, 6, 4, 9, 2, 7, 5, 0, 3, 1 needsto be sorted using the Quick sort algorithm. Illustrate thesorting algorithm behavior by producing a new sequenceafter each element exchange and specially mark thenumbers relevant for the next step (exchanges, pivotalelements, median approximation). The pivotal elementneeds to be chosen with the method of median approxi-mation of the rst, middle, and last sequence element.Figure 8 User_Action_Points formula graphs for the two distinctAlg_Weight constants.Table 2 Initial Survey Items and Summarized Students ResponsesLikert Scale Question Items (N 122)Survey item AVG SDBefore the sorting algorithms lectures started I already knew sorting algorithms well 1.98 1.083During the ASP course lectures I am able to learn course contents well 3.17 1.197I learn the most by studying literature (e.g., slides, books, textbooks, exams) at home 3.88 1.182I learn the most when studying with colleagues in a group 2.66 1.270I like the idea of using technology in learning programming 4.28 0.884I believe the technology can help in improving education 4.62 0.647I believe the technology can be used for learning 4.56 0.793I believe there are advantages to using smartphones in education compared to classic desktop computersdue to their characteristics (easy to carry, instant connectiveness, etc.)2.99 1.269I believe sorting can be learned with the help of computers 4.41 0.747I believe sorting can be better learned with the use of computer than without the use of computers 4.29 0.922I would like to participate in the research on learning sorting algorithms using smartphones 3.09 1.477Table 3 Initial Survey Items and Summarized StudentsResponsesYes/No Question Items (N 122)Survey item Yes NoI own a mobile phone (of any kindcellphone, smartphone, iPad, etc.)120 (98%) 2 (2%)I own a smartphone (Android, iPhone,Windows Phone 7, Blackberry, etc.)47 (39%) 75 (61%)I own a smartphone with an Androidoperating system29 (24%) 93 (76%)6 BOTICKI ET AL.In our study we were interested in nding out the correla-tion between the three main data sources: the total points stu-dents had on the Sortko system after the semester was done, thetotal points students had on the Quick sort algorithm on theSortko system after the semester was done, and the results fromthe above presented assignment (standard midterm exam task4). The correlations are given in Table 4.Since the standard exam assignment included Quick sortalgorithm, we calculated the correlation between the applicationusage frequency and the standard exams assignment results andgot a fairly high value of 0.47 with a signicance of P < 0.05.Furthermore, the correlation between the Sortko total rank andthe Quick sort assignment results gives a similarly high valueof 0.53. Therefore, we became condent that Sortko usagemeans better standard exam results and explored this hypothesisby applying the independent sample t-test to compare betweengroups according to the mean values of usage frequency, totalSortko points, and the Quick sort Sortko points (Table 5).The analysis shows there is a signicant difference be-tween exam results when comparing Sortko users and Sortkonon-users of around 30% according to all three criteria. Theeffect was large-sized of r 0.52, 0.70, and 0.65 for total us-age frequency, total Sortko points, and Quick sort Sortko pointsrespectively. As a control mechanism, there are no signicantdifferences between the Sortko users and Sortko non-userswhen comparing their second midterm and rst and secondmidterm exam results. This proves our initial hypothesis thatSortko users achieve better standard academic results thanSortko non-users.The Final SurveyThe nal survey was done after the lectures and the secondmidterm exam were completed. It was conducted on a volun-tary basis attracting 29 students who used the Sortko system.They were asked how practical the application is when learningsorting algorithms, how well individual application moduleswere structured, etc. (Table 6). Additionally, as a part of thesurvey, we were interested in nding out whether and how oftenour students used the application. Sixteen out of 29 studentsclaim they used Sortko once, while 27 out of 29 responded theyused it more than once during the semester.In the analysis of the nal survey, we separated the stu-dents into two groups according to the academic success andthe usage frequency criteria. We wanted to identify surveyitems for which these groups of students gave signicantly dif-ferent responses.Generally, the students feel Sortko helped them in learningsorting algorithms and in achieving better results in standardexams which is also conrmed by the quantitative analysis pre-sented in the previous section. They feel the Sortko Scaffoldingmodule helped them in dealing with doubts while sorting, whilethe motivational module attracted a diverse set of responseswhich is analyzed in detail in the following sub-section.In order to further elaborate on the students answers tothe survey questions, they were separated into two groupsaccording to two main criteria: Their academic success andtheir Sortko usage frequency. By doing this we were able toexplore whether the difference in students success and thenumber of times they used Sortko affects their perception asexpressed through the administered survey. When comparingstudents according to their academic ability we found out thereis a signicant difference between more and less academicallysuccessful students on only one survey item: The Sortko help message system helped me to learnsorts better (P < 0.05, one-tailed signicance level,r .19, small-sized effect).When comparing students answers to the survey questionsaccording to the Sortko usage frequency, we found out there isa signicant difference between students who used the applica-tion a lot and the ones who used it less on the following surveyitems: Sortko application helped me in learning sorts(P < 0.05, two-tailed signicance level, r .40, large-sized effect). Sortko application helped me in achieving better resultson exam/quizzes/short lecture assessments (P < 0.05,two-tailed signicance level, r .33, medium-sized effect). The Sortko rank list motivated me to compete and usethe Sortko application multiple times (P < 0.05, two-tailed signicance level, r .70, large-sized effect).Motivational Module Effectiveness AnalysisSortkos motivational module was used to introduce a game ele-ment into the sorting activities. The rewards system was set upso that it stimulates using more complex algorithms such as theQuick sort or Shell sort by simply awarding such sorts withmore points compared to the simpler algorithms.Table 4 Correlations Between Sortko Points, Sortko Usage, andMidterm Exam ResultsTotal SortkopointsQuick sortSortko pointsMidterm 4thexam pointsQuick sort sorting frequency 0.751 0.802 0.473Total Sortko points 0.788 0.528Quick sort Sortko points 0.499P < 0.05 (two-tailed).P < 0.01 (two-tailed).Table 5 Independent Sample t-Test Exploring the Difference in Academic Results Between Sortko Users and Sortko Non-Users (N 24)Total usage frequency Total Sortko points Quick sort Sortko pointsFourth standard exam results (mean differencebetween groups in % of the assignment total)29,076 36,333 31,143Second midterm standard exam 4,711 3,833 3,657First and second midterm exam results 3,956 3,167 3,171P < 0.05 (two-tailed).SORTING WITH MOBILE DEVICES 7The correlation of students responses on the survey ques-tion Rank list motivated me to compete and use Sortko multi-ple times (AVG 4.29) and the usage frequency for thestudents who actually used the application is r .35, present-ing a medium-sized effect. Sortko power users gave more posi-tive answers to this question which conrms the rewardssystem motivated students to compete and use Sortko multipletimes.An interesting issue arises in the nal survey analysiswhich shows that the application use frequency negatively cor-relates with the perception of adequacy of the rewards system(r .37). However, by taking into account the nal surveyresults analysis on the question item The Sortko rank list mo-tivated me to compete and use the Sortko application multipletimes, we were able to identify there is a signicant differenceof 1,894 between frequent and non-frequent Sortko users(P < 0.001, two-tailed signicance level). The explanation forsuch responses was looked for through informal personal inter-views in which we found out that in some cases points getawarded depending on luck. The main reason for that is theinitial sorting sequence generation algorithm which in somecases generates sequences which are easy to sort. This leads toan interesting situation where a couple of Sortko power usershad an extremely negative opinion regarding the rewards sys-tem. Regardless of the perceived unfairness, the correlation co-efcient between the usage frequency and the perceivedmotivational effects of the rewards system is relatively high(r 0.33).A surprising effect of the motivational module emergedafter the nal survey was administered and the study was for-mally completed with additional 519 sequences sorted (27% ofthe totally sorted sequences). In addition to the pre-exam time,there is a usage peak between May 12, 2011 and May 22, 2011,a period after the nal exam (Fig. 9). This sustained usage tellsus the standard exam is not the only motivational element forusing Sortko and conrms the benets of the Motivationalmodule.CONCLUSIONSThis article described a case study exploring the benets ofusing mobile devices in supporting teaching and learning ofsorting algorithms, an important computer science topic. Weproposed an approach in which an Android mobile applicationwas designed and made available to students to be used in andout of classrooms. As a contrast to the conventional mode ofteaching and learning, this application allowed students to en-gage in learning sorting algorithms in a location-independentwayboth in and out of classrooms. By leveraging the existingbody of knowledge dealing with the use of animation in teach-ing and learning, we designed an interactive application whichscaffolds students in trying out sorting on their own. Our designTable 6 Final Survey Items and Summarized Students ResponsesLikert Scale Question Items (N 29)Survey item AVG SDSig.(two-tailed)ac. succ.Meandiff.ac.succ.Sig.(two-tailed)usage freq.Meandiff.usagefreq.Before the lectures on sorting in ASP course started I knew sorts well 1.86 0.953 0.128 0.543 0.850 0.071Sortko application helped me in learning sorts 4.28 0.922 0.735 0.119 0.037 0.727Sortko application helped me in achieving better results onexam/quizzes/short lecture assessments3.86 1.246 0.547 0.286 0.002 1.247By using Sortko application throughout a longer period of time,the sorting course contents are better mastered4.21 1.146 0.548 0.262 0.220 0.545The Sortko help message system helped me to learn sorts better 4.31 0.761 0.098 0.462 0.435 0.232The Sortko help message system helped me in clearing somedoubts regarding sorts4.00 0.845 0.192 0.414 0.179 0.439Sortko help messages clearly direct on the application usage andidentify errors during sorting3.72 1.066 0.466 0.295 0.490 0.288The Sortko rank list motivated me to compete and use theSortko application multiple times3.55 1.639 0.778 0.176 0.000 1.894The Sortko reward (points) system is adequate and fair 3.48 1.379 0.950 0.033 0.428 0.485The Sortko reward (points) system motivates me toadditionally use the application3.72 1.306 0.382 0.433 0.384 0.444I like Sortkos graphical design 4.45 0.827 0.751 0.100 0.159 0.449The Sortko application is easy and intuitive to use 4.34 0.936 0.947 0.024 0.195 0.470Now many points did you collect at the ASP course so far? 33.81 10.47P < 0.05 (one-tailed).P < 0.05 (two-tailed).P < 0.001 (two-tailed).Figure 9 Sortko usage peaks (summer 2011 semester).8 BOTICKI ET AL.acknowledged the role of motivation in learning by awardingstudent with points which were their accumulated on the com-mon rank list stimulating prolonged application use.We found out that student like using technology for learn-ing, although they are not all sure that smartphones specicallycan help in learning. However, we were able to nd out that thestudents who already own a smartphone recognize its educa-tional benets better than the non-owners. As far as the Sortkosmotivational module is concerned, we are able to report thatour students found it effective and that it enticed students on aprolonged system use. On the other hand, standard exam resultsanalysis shows Sortko users achieve better results compared toSortko non-users.We plan to upgrade Sortko for the upcoming academicyear, focusing on improving the Motivational module by reduc-ing the luck factor in the initial number sequence generation aswell as creating a skill-based ranking mechanism in addition tothe purely cumulative one. This should make the Motivationalmodule more effective in motivating students on using Sortkoand neutralize the annoyance exhibited by power users who fa-vor skill-based competition, as shown by the analysis. Due tothe rapid growth of the smartphone market porting Sortko ontoother prevailing mobile platforms, such as iPhone or WindowsPhone 7, should allow us to reach wider audiences and afrmSortko as an educational training tool for mastering sortingalgorithms.Although the application and the results were encouraging,we cannot emphasize the role of curriculum integration more.Since the Sortko application and this study were done as anadd-on onto the curriculum, we were not able to motivate themajority of students to participate. It is our belief that similareducational interventions have to be employed as well integrat-ed curriculum elements and not as add-ons to achieve the fulland sustained educational power.To sum up, mobile learning applications such as Sortkoare able to engage students and can serve as a motivationalvehicle in computer science engineering motivation. Studentsare open to such interventions and educators should leverage onthe intrinsic benets of mobile technologies in developing theirteaching and assessment strategies.REFERENCES Android-Online, Android developers guide, Retrieved February15, 2012, From http://www.android.com/developers/. L. Naismith, P. Lonsdale, G. Vavoula, and M. Sharples, Literaturereview in mobile technologies and learning, report no. 11, Future-lab, Bristol, United Kingdom, 2005. H. B. Andreu and M. Nussbaum, An experimental study of theinclusion of technology in higher education, Comput Appl EngEduc 17 (2009), 100107. J. Roschelle, Keynote paper: Unlocking the learning value of wire-less mobile devices, J Comput Assist Learn 19 (2003), 269272. C. Romero, S. Ventura, and P. de Bra, Using mobile and web-based computerized tests to evaluate university students, ComputAppl Eng Educ 17 (2009), 435447. G. Vavoula and M. Sharples, Challenges in evaluating mobilelearning, Proceedings of the mLearn 2008 Conference, Shropshire,United Kingdom, 7th October 2008. M. Ebner and A. Holzinger, Successful implementation of user-centered game based learning in higher education: An examplefrom civil engineering, J Comput Educ 49 (2007), 873890. A. A. Deshpande and S. H. Huang, Simulation games in engineer-ing education: A state-of-the-art review, Comput Appl Eng Educ19 (2011), 399410. M. Milrad and M. Jackskon, Designing and implementing educa-tional mobile services in university classrooms using smart phonesand cellular networks, Int J Eng Educ Mobile Technol Eng Educ23 (2007), 8491. T. A. Philpot, R. H. Hall, N. Hubing, and R. E. Flori, Using gamesto teach statics calculation procedures: Application and assess-ment, Comput Appl Eng Educ 13 (2005), 222232. A. Drozdek, Data structures and algorithms in C, Course tech-nology, Boston, 2005. M. A. Weiss, Data structures and algorithm analysis in C, AddisonWesley, Reading, MA, 1997. M. D. Byrnea, R. Catramboneb, and J. T. Stasko, Evaluating ani-mations as student aids in learning computer algorithms, ComputEduc 33 (1999), 253278. S. R. Hansen, N. H. Narayanan, and D. Schrimpsher, Helpinglearners visualize and comprehend algorithms, Interact MultimediaElectron J Comput Enhanced Learn 1 (2000). C. Kann and C. Hundhausen, Integrating algorithm visualizationtechnology into an undergraduate algorithms course: ethnographicstudies of a social constructivist approach, Comput Educ 39(2002), 237260. R. M. Baecker, Sorting out sorting. Film. University of Toronto,M. Kaufmann Publishers, San Francisco, USA, 1981. B. Deliktas, Computer technology for enhancing teaching andlearning modules of engineering mechanics, Comput Appl EngEduc 19 (2011), 421432. H. L. Dershem and P. Brummund, Tools for web-based sortinganimation, ACM Special Interest Group Comput Sci Educ BullInroads 30 (1998), 222226. T. L. Naps, Algorithm visualization in Computer Science Labora-tories, Proceedings of the 21st SIGSE Technical Symposium onComputer Science Education Special Interest Group for ComputerScience Education Bulletin Inroads, Washington, USA, February1990. E. Sutinen, J. Tarhio, and T. Tera, Easy algorithm animation onthe Web, Multimedia Tools Appl 19 (2003), 179194. M. Kordaki, M. Miatidis, and G. Kapsampelis, A computer envi-ronment for beginners learning of sorting algorithms: Design andpilot evaluation, Comput Educ 51 (2008), 708723. S. Price and Y. Rogers, Lets get physical: the learning benets ofinteracting in digitally augmented physical spaces, J Comput Educ15 (2004), 169185. D. R. Martin, Sorting algorithm animations, 2007, From http://www.sorting-algorithms.com/. G. G. Robertson, S. K. Card, and J. D. Mackinlay, Informationvisualization using 3D interactive animation, Commun ACM 36(1993), 5771. J. S. Gurka, Pedagogic aspects of algorithm animation, Unpub-lished PhD Dissertation, University of Colorado, Colorado,1996. J. Stasko and E. Kraemer, A methodology for building applica-tion-specic visualizations of parallel programs, J Parallel DistribComput 18 (1993), 258264. D. Bornstein, Presentation of Dalvik VM Internals, Google,2008. Eclipse-Online, About the Eclipse Foundation, Retrieved February15, 2012, From http://www.eclipse.org/org/. S. F. Allam, SOA Design pattern for distributing the object modelin n-tier applications, Innovations and advanced techniques in sys-tems, computing sciences and software engineering, Springer,Netherlands, 2008, pp 182185. FER-ZPR, Sortko website, 2012, From http://nihao.fer.hr/sortko(alternate url http://188.8.131.52/Sortko). FER-ZPR, Sortko Android market web page, 2012, From https://market.android.com/details?idfer.sortko.com.SORTING WITH MOBILE DEVICES 9BIOGRAPHIESDr. Ivica Boticki is a researcher and lecturerin the Department of Applied Computing,Faculty of Electrical Engineering and Comput-ing, University of Zagreb, Croatia. Hisresearch interests include mobile learning,software engineering and engineering educa-tion. He teaches courses in software engineer-ing and does research in technology enhancedlearning focusing on two main areas: primaryschool education and the enhancement of edu-cational practices in software engineering education. He is an authorand co-author and reviewer of publications in several journals andconferences.Ante Barisic got his masters degree in com-puter science and is currently pursuing anothermasters degree in economics at the Universityof Zagreb, Croatia. Passionate about productdesign and development, he started his careerwith developing ICT products in telecomindustry. His main area of interest is the devel-opment of web and mobile applications.Dr. Sergio Martin is an assistant professorand researcher in the Electrical and ComputerEngineering Department at UNED (SpanishUniversity for Distance Education) since 2006.He has a PhD in Computer Engineering. Hehas participated since 2002 in national andinternational research projects.Neven Drljevic is currently pursuing his mas-ters degree in computing at the University ofZagreb. Enthusiastic about new technologiesand opportunities they open up for youngpeople, several years ago he started the WebStart Contest, a successful local competitionand educational course in developing studentweb start-up projects, organized by the studentassociation eSTUDENT.10 BOTICKI ET AL.