Showcase

Virtual Reality

Seaweed Heroes

Seaweed Heroes (http://www.seaweedheroes.com) is a unique psychedelic adventure multiplayer Virtual Reality game I developed in 2019 in collaboration with an artist friend using Unity 3D. The game features: unique locomotion system, multiplayer, spatial audio, AI system and conversational agent using IBM Watson Natural Language Processing. I am currently exploring self-publishing via Early Access on Steam.

Steampunk Sky Pirates

Steampunk Sky Pirates is a virtual reality game prototype I developed in 2016 using Unreal Engine 4, Oculus Rift DK2 and Leap motion. Leap motion is used for hand tracking and player input for rocket propelled flight. Flight behaviours for enemies and allies (based on Craig Reynolds' Flocking behaviours) were developed in c++ exposing functionality to Blueprints.

Monster Maze

Monster Maze is a virtual reality game prototype I developed in 2016 using Unreal Engine 4, Oculus Rift DK2 and Leap motion. Leap motion hand tracking is used for combat and player movement. The mazes are procedurally generated using c++ and exposed for Blueprint implementation.

Gamification

I collaborated with academic staff from 2013 to 2016 on the iterative participatory design and evaluation of the gamification of the ‘Computer Game Development’ course offered to 2nd year students. The gamification aims to support course objectives by improving students’ review of course material, increasing meaningful class participation, fostering problem solving skills, increasing lecture attendance and encouraging creativity in practical assignments. This was achieved by enhancing the existing course using various gaming features (narrative, characters, puzzles, quizzes, leader board and rewards) towards learning objectives, motivational affordances, engagement and behavior change.

We found that the gamification has a positive impact on student engagement with the course. It increased lecture attendance, class participation and student enjoyment. Student engagement in learning activities is a key indicator of high quality learning. Observations suggest that student agency is crucial for a positive emotional response and necessary for students to buy in and realize the learning activity. Digital media can provide such opportunities for learning. However, digital media is assumed to be inherently motivational. This assumption is not empirically supported. I presented this and more at the 49th Hawaii International Conference on System Sciences in January 2016.

Educational Games

Enforcer Gauntlet Nero-evolution Tournament (2014 - 2016)

Students find Artificial intelligence challenging. They often consider it as difficult ‘black box’ technologies and mathematical formalisms. I developed the Neuro-evolution Tournament for the 3rd year ‘Machine Learning’ course in consultation with the Artificial Intelligence (AI) lecturer. The system scaffolds the complex algorithms of Q-Learning and NeuroEvolution of Augmenting Topologies (NEAT) through an interface with sliders that players can use to configure the fitness functions used to train agents. The agents’ behavior and performance is also clearly apparent in the 3D visualization. The training of AI agents and tournament simulation uses the OPENNERO AI research framework. It is a general-purpose open source platform that aims to simplify the process of implementing machine learning games and of conducting AI experiments. It has been used as a research tool; however, its educational utility still remains to be proven.

Our tournament builds on OpenNERO. Players compete for rank and prizes through personal assessments and the performance of their teams of agents. The tournament is organized around 3 missions and allows for the pre- and post-test evaluation of learning. The missions present players with 3 maze solving problems of varying complexity. Mission 1 requires players to solve a set maze by training a group of simulated robotic agents. Players develop the fitness function to train agents to solve the maze in the shortest time and avoid local minima. Mission 2 complicates the maze by introducing static enemies and the cost of damage suffered by agents. Mission 3 introduces moving enemies and random mazes, requiring students to evolve generalized maze solving agents.

Students reported developing a better understanding of the relationship between the fitness function and the agents’ behavior. They also learned that agents’ behaviors are best tuned over many iterations. However, we did not have enough participants completing the tournament to draw conclusive results. A major stumbling block was the time required to train the agents as result of the computational demand of the NEAT algorithm.

A* Pathfinder Serious Game (2013 - 2016)

We found that students often understood the A* pathfinding algorithm conceptually. However, in practice the recursive steps of the algorithm were not performed systematically. I developed the A* Pathfinder serious game to provide students with a fun drill-and-practice exercise of the algorithm. In the game story Dr. Gerasimov (a scientist that found himself as a head in a jar after a near fatal accident) requires the player to find mechanical components in the maze like laboratories to build a mechanical body for him. The A* Pathfinder Mechanism (a compass like user interface element) scaffolds the learning of the A* Pathfinding algorithm. It is presented in the game story as an ingenious machine that calculates a safe (shortest) path through the laboratories and their traps. As levels progress the Pathfinder malfunctions, requiring students to manually set values. Through this the scaffolding of the learning is gradually removed. When students make mistakes, they set off traps that depleted their health serum and score. Dr. Gerasimov provides feedback to the player. This immediate feedback is highly valued by students and facilitates their mastery of the algorithm. Students report that they found it easier to visualize the execution of the A* algorithm as it was stepped through using the dynamic and interactive serious game. They found the visual aspects and animations of the game appealing. Students rated the immediate feedback and the Pathfinder Mechanism highly valuable when considering what contributed to their learning. The initial iteration of the experimental method for evaluating learning in serious games did not provide statistical evidence for these claims. However, the second iteration provides statistical evidence that students made significantly less errors in a post-test exercise.

Eskom Energy Planner (2012)

Eskom is the national electricity provider of South Africa. The online game was designed to educate players about the different power generation technologies in an entertaining way. It allows the player to take custody of a virtual city’s power plan and seek a balance between the most cost effective technologies currently available and the most environmentally friendly ones. I was the lead programmer and coordinated the development of the game in collaboration with junior programmers. I also worked closely with Engineers from Eskom to design the internal economy of the game. It was nominated for MOST SIGNIFICANT IMPACT category at the Games for Change Festival in 2013.

IBP Game (2010)

The International Budget Partnership collaborates with civil society around the world to use budget analysis and advocacy as a tool to improve effective governance and reduce poverty. We developed a serious game in collaboration with budget efficacy experts and workshop facilitators. In the scenario-based learning game, players assume the role of an intern working for a budget advocacy organization. Their objective within the game is to improve the lives of poor and marginalized people.

NANO Game(2011)

This touch screen game was developed for the Cape Town Science Centre. I developed the instructional design, architecture for the game system and managed the development of content by junior developers. The NANO game is an educational touch screen application, which informs learners on the basic principles and practical application of nanotechnology. The application is structured into three sections: 1. Explains the principle of how nanotechnology can increase the surface area of matter. 2. This principle is applied to a real-life scenario, where the user is challenged to re-engineer a cell phone battery using nanotechnology. 3. Informs the user on careers in the field of nanotechnology.

Tanzanite Experience (2012)

The Tanzanite Experience is an interactive shopping experience focused around the sourcing, extraction and shaping of this semi-precious gem stone, Tanzanite. The experience includes 3 multi-media touch screen games that inform visitors of tanzanite mining, preform selection, cutting and polishing.

Course development and Teaching

The Computer Game Development – Bachelors in Computer Science (Undergraduate)

This course aims to provide 2nd year students with knowledge, skills and practice of game development in alignment with industry requirements. This course introduces high-level game programming concepts. However, theory is transformed into skills and knowledge through constructivist learning activities. These include workshops, guest lectures from industry professionals and practical assignments to realise the concepts covered. The major deliverable of each student is a 2D game, which production is scaffolded through practicals.

I was responsible for the “Game Artificial Intelligence” module (for years 2013 and 2014). In 2015 I was responsible for the “Playability and Design ” module and the Pathfinding section of the “Game Artificial Intelligence” module. In 2013 we had 55 students enrolled in the course, in 2014 we had 67; and 2015 we had 46.

Mobile Game Development - Honours Degree in Computer Science (Postgraduate)

The aim of this elective module is to provide 4th year students with a deep understanding of the technical aspects of Computer Game Design, and a broad understanding and appreciation of the collaborative nature of the design and development process of a successful game. The module is structured around the collaborative learning activity of working in a team and completing a mobile game from the phases of conceptual design through to final playtesting on the target device (mobile phone or tablet).

I participated in the development and implementation of this module in 2014 and 2015. In 2014 we had 27 students enrolled and in 2015 we had 15. The module is oriented around the collaborative group task to produce a mobile game over a period of two months. The task is scaffolded through iterative deliverables, including: Game pitch, Interaction design, Prototype, Games design document, Technical design document and Portfolio. Shortly after the submission of each deliverable the class comes together for a discussion and feedback session. Groups present their deliverable and receive feedback, suggestions and questions from lecturers and peers. They are then expected to revise the deliverable in light of the feedback session.

Principles of Design for Interactive Media - Bachelor of Arts in Film and Media Production

I developed and taught this course in 2006 while completing my PhD. The course prepared 3rd year film and media students for the production of Interactive Media with an understanding of the available technologies, processes and methodologies that are used in the professional industry. The course had 13 participants and ran from February to May; with two three-hour classes a week. Students participated in practical activities using the relevant technologies and methods. The course provided an understanding of principles as well as an insight into many specific areas of practical application).

Responsive Web Development - Bachelor of Arts in Film and Media Production

I participated in the development and teaching of the ‘Interactive Media’ course in 2012 while working as an independent consultant. The course introduced 2nd year film students to essential concepts, techniques, practical skills and knowledge required for the production of interactive media on the Web. Students designed, created and assembled standards-compliant websites and created usable and aesthetically pleasing designs from original visual materials. They were given an overview of the principles and concepts needed for working with the major components of digital media (images, text, video, and audio) and were introduced to some key applications needed for producing Web-based media. Students were expected to produce and maintain a standards-compliant portfolio website, prepare a short online video and write a feature article suitably edited for online publication. I was responsible for the ‘Responsive Web Design’ module of the course.

E-learing

Distell eLearning

Distell was created in 2000 by the merger of Stellenbosch Farmers' Winery and Distillers Corporation. In 2014 I collaborated on the development of a series of 7 e-learning courses for the professional development of Distell’s laboratory staff. The courses support Distell’s in-house training and blended learning approach. I implemented the interface, menu structure, tracking and interactions, including: software simulations, interactive puzzles and video clips. The modules were packaged according to the SCORM standard for their Moodle LMS (http://distell.ilite.co.za/).

British American Tobacco South Africa

  • IBM LearningSpace5 installation, administration and support
  • Knowledge transfer workshop regarding IBM LearningSpace5
  • Developed and implemented the E-learning "Orientation Program" (part of BATSA's change management strategy) using Macromedia Authorware 6

Eskom

I participated in the development of the e-learning Strategy of the ESKOM power utility. This strategy development included an e-learning pilot study to demonstrate the return of investment for the blended learning approach. The IBM LearningSpace 5 LMS was used for the e-learning pilot. Online e-learning materials was selected to supplement Eskom’s face to face training through a blended approach. The pilot successfully demonstrated the return of investment this approach.

  • Programmed the interactions, tracking, games, assessments and the SCORM communications with the Learning Management System for the "Quality Management", "Conditions of Service" and "Human Performance" courses using Macromedia Flash MX
  • Designed and developed the communication web site utilized to communicate with stakeholders regarding the E-learning Strategy developed for Eskom

Daimler Chrysler South Africa

I participated in a blended learning pilot study at Daimler Chrysler South Africa. I facilitated the IBM LearningSpace5 installation, administration and support. I also developed an online application for assessing learners' experiences of the system.

Santam

Santam is a large insurance company. It developed a new Data Warehousing Intranet Application and dashboard. Santam required the development of an online course to facilitate their staff to learn the new system, skills and processes. I participated in the development and implementation of this course using Macromedia Authorware 6.5. The course supplemented existing face to face training. A systems simulation was developed as part of the course to afford staff the opportunity to practice in a safe environment.

Department of Justice

The Department of Justice of South Africa required e-learning to supplement their staff training, in particular with regard to computer literacy. I participated in the curriculum planning, instructional design, story boarding, screen capturing and authoring of course material (using Macromedia Authorware 6) for Basic Computer skills, Windows 2000, MS Word XP, MS Excel XP and MS Outlook XP.

Virtual Learning Environments

Film Production - Collaborative Learning Environment

In 2003 I attained a competitive scholarship from the Collaborative African Virtual Environments System (CAVES) project at the Collaborative Visual Computing (CVC) Laboratory, which aims to develop methods and systems for collaborative visualization. My PhD research contributed towards this objective by developing two virtual environments (using the Unreal Engine 2.0). The systems aim to facilitate collaborative problem-based learning activities in the domain of film production.

Film production is a collaborative endeavor. It requires careful planning and collaboration between the members of a production team. Film students lack experience as members of such teams. The Virtual Film Production System is a networked virtual environment developed with the aim to provide students with a well-developed shared virtual world to contextualize and facilitated the planning and production of a Long Take shot. The scenario was adapted from a Long Take learning activity at the Centre for Film and Media Studies. It required careful planning and choreography between the camera, actor, props and cinematic space.

The system scaffolds the task through defining roles explicitly and providing role specific functionality. The roles of director, camera operator, and actor are supported. Video analysis and interviews with students reveals that the system benefitted the learning activity, facilitating creativity within constraints and providing an information rich context to develop a plan and choreography for the shot.

Urban Design - Collaborative Learning Environment

In 2000 I attained a competitive scholarship from the ‘Stichting Studiefonds voor Zuidafrikaanse Studenten’ to complete a Master of Science Education and Training Systems Design. My MSc project developed a networked virtual environment with the objective to support a collaborative problem-based learning activity in an online Urban Design educational scenario.

Collaborative problem definition and goal setting is challenging. The Urban Design problem space is not clearly represented only in drawings and maps. It is difficult to collaboratively visualize, negotiate and manipulate the space using these representations online. The virtual environment provides scaffolding for the collaborative visualization of the urban design problem, which collaborators can inhabit, navigate, organize, negotiate and manipulate. It also supports the activity and solution creation by assigning roles, resources and tasks to participants. The artifact produced and a discourse analysis revealed that the system successfully facilitated the collaborative problem definition and goal setting aspects of the problem-based learning activity.

Interactive Wall Projections

Leopard’s Leap

Leopard’s Leap Wine Farm commissioned Formula D interactive to design and build an educational leopard display for their visitor centre in Franshoek. The 3-meter interactive wall projection uses the Microsoft Xbox 360 Kinect controller as we found it more reliable than the infrared camera tracking system. When visitors step in front of the wall, they are detected and virtual avatars appear on the wall. Players can trigger different hotspots on the wall, triggering text information, video and audio. Sound effects and ambient noises are played over an overhead directional speaker.

Gravity Game

The game was developed for the Sci-Bono Discovery Centre. The gravity game is an interactive wall projection physics simulation that enables users to play with 5 types of virtual balls (soccer, basket- ball, tennis, table tennis and golf) in three different gravitational environments: normal gravity (earth), low gravity (moon) and high gravity (planet x). When users move in front of the wall, the shape of their bodies is represented by a blue silhouette on the projection. The aim of the game is to keep the ball up in the air by juggling it with your feet and doing this in different environments demonstrates the principle of gravity. The system uses infrared lights, camera and the OpenCV framework [6] to track the user’s silhouette. The OpenCV polygon silhouette is exposed to the Adobe Flash application to represent the player in the visualization. It also provides a collision shape for the virtual balls physics system.

Multi-touch Interactive Displays

Formula D interactive

Formula D interactive employs a variety of sensor technologies for its custom-design touch tables and interactive museum exhibits. I participated in the development of 2 large multi-touch displays. The displays use rear projection and Frustrated Total Internal Reflection technology, trapping Infrared light inside an acrylic screen.

When a user touches the screen, Infrared light leaks through the touch points. The positions of the points are captured with an Infrared camera. We used the TUOI framework to process touch points and communicate them to Adobe Flash creative media applications. The applications support using natural gestures, touching, scaling, rotating and sliding virtual objects across the display with one or both hands. The system enables multiple users to interact simultaneously with the application, facilitating a collaborative experience.

I participated in the upgrade of the 1.5 x 2 meter Interactive ‘Frog Touch Wall’, an educational media space at the Two Oceans Aquarium (http://www.aquarium.co.za).

I also participated in the development of the ‘Public Service Explorer’ a 2 x 2 meter multi-touch application for the Centre for Public Service Innovation (http://www.cpsi.co.za). The application enables multiple users to visualize and collaboratively innovate flow charts of processes used in public service.

Web Development

SEAGRAM DATAMART APPLICATION 2000

OTUK was contracted by Seagram Spirits And Wine Group to review data warehousing covering its European distribution enterprise. OTUK and seagram partnered in the development of an Internet-enabled plug and play data mart solution. The mart mart comprises Intel-based server hardware, the Harvester data-pump for scheduled transfer and transformation of data, a minimal operating system (OS) layer; Dimensional Insight's Diver for enterprise wide data visualisation, analysis and reporting, and Webdiver for the remote interrogation of data. [NEWS/400.uk]

In 2000 I developed the SEAGRAM DATAMART APPLICATION, a web application uses Active Server Pages, Oracle 8i Database and the AspEventViewer Component running on IIS (Internet Information Server). The application enables Administrators to create Users, Roles, Models and Types. These structures enables access to data models according to rules. The application uses SQL queries to interrogate the Oracle Databases, hosted on various servers, allowing the administrators to remotely access and diagnose Databases. I developed a web-clipping view for the Palm Pilot VII handheld computer and Integrated Dimensional Insight’s Diver BI java applet for exploring the data through a web browser.