Stanford University x Seeed: Open Source Haptic Device for Affordable Haptics Education for All

Project Name: Hapkit Project

Deployment Location: California, USA

Targeted Industry Type: Education

Project Partner(s):

Haptics implies the sense of touch, and its hands-on learning tool is called haptic devices. When we think about our sense of touch, we might not put as much importance as hearing (ears), seeing (eyes), or smelling (nose) functions of the human biological functions. However, with the rise of the 4th Industrial Revolution, haptics is getting more and more attention from science, technology, and various industry players for new opportunities. Unfortunately, not everyone can learn about the technologies embedding haptics, as haptic gears can be oftentimes expensive and difficult to assemble. This is why, many academic experts have been researching, experimenting, and devising inexpensive, easy-to-assemble, and open source haptic devices for many years (Martinez et al, 2020).

Now, if you think about various games applying AR technologies, or immersive movie experience, all with embedded haptic devices, you might be able to guess why we want to talk about the haptics today: the essence of touch, and its potential for solving a lot of social challenges in our world society (Figure 1).

 Figure 1. What Haptics Looks Like in Reality (Rappler, 2015; Tran, 2017)

What’s the Challenge?

How to make haptic devices more accessible and affordable for educational purposes?

What’s the Project About?

In 2012, Prof. Allison Okamura from the School of Mechanical Engineering, and Prof. Paulo Blikstein from the School of Education at Stanford University came together to devise upgraded educational haptic paddles used for teaching system dynamics, control theory, and mechatronics concepts. With a goal of disseminating the usage of the haptic devices for big-scale educational projects, they created a DIY electronic kit called “Hapkit” for on-campus courses, online courses and educational research (Figure 2). This Project is designed for such educational projects, in which mechanical engineering students and other multi-stakeholders from all educational and professional backgrounds can work together for collaborative projects, that involve programming and simulating the sense of touch on computer-based virtual environments. For example, when a learner touches the haptic device, and makes movements, they can feel “what it’s like to run his or her hand against a wall, or click a ballpoint pen” (Ramachadran, 2015).

Now, the multi-stakeholders might not be used to harnessing haptic devices in the past, so this easy-to-assemble, and easy-to-learn “Hapkit” can enable them and the students to grasp scientific and technological knowledge more easily (Hapkit, n.d.). For this Project, they worked with Seeed in 2013 to design and make an all-in-one controller board called “Hapkit Board” to reduce the costs on microcontrollers and amplification that supports the Hapkit.

Figure 2. Hapkit Developed at Stanford University (Hapkit, n.d.)

Prof. Allison Okamura is an Associate Professor at Stanford University, teaching dynamic systems, robotics, haptics, and medical technology at the mechanical engineering college. Her academic expertise lies in haptics, medical robotics, teleoperation, virtual environments and simulators, prosthetics, neuromechanics and rehabilitation, and engineering education. Prof. Okamura is also the Director of the Collaborative Haptics and Robotics in Medicine (CHARM) Lab at Stanford, where she creates gadgets necessary for materializing advanced robotic and human-machine systems that are able to interact with physical touch. During her career, she has been a Co-Chair of the IEEE Haptics Symposium. By and large, IEEE is known as “the world’s largest technical professional organization dedicated to advancing technology for the benefit of humanity”, which shares similar values with Prof. Allison Okamura and Seeed Studio.

As the word suggests, “haptic devices” are enabled by the human’s sense of touch (Vanderbilt MEDLab, 2013). Now, many of you might be unfamiliar with the concept of haptic devices, so here’s how Prof. Okamura defines it in plain wordings (Tedx Talks, 2013):

“When you type on your computer, or you play on your iPad or your smartphone, … your interaction [with those devices] is more on the touch side of  creating inputs to the device: ‘you’ touch it, but ‘it’ doesn’t touch you back. … Science and technology of haptics is about creating an interface, where you can really interact in a more profound way with those kinds of devices.”

“Now, [the] basics behind this technology is that, you record users’ movements: You see how they behave … by [letting them] holding onto a little robotic device. And that device records the motions, and can transmit that to the movement of … a cursor in a virtual environment [=computer screens] (Figure 3). Now, if I were a haptic programmer … writing codes in this virtual environment, I could say [=write commands] that, if I touch this teapot, then the user should feel a force from the teapot. Therefore, I’m going to program my device to have motors that push back on the users, so that you [=they] can feel what’s going on in that virtual environment.”

Figure 3. How a Simple Haptic Device Works  (Tedx Talks, 2013)

Hapkit is a low-cost  (USD 50) open-hardware haptic device, which allows users to input motions and then feel programmed forces. The Hapkit Board (Figure 4) is as small as Arduino Uno, and merges a microcontroller Atmega328, Micro SD card slot, and Motor driver L298P. This combination enables the Hapkit to achieve 3 functions: control operation, data recording, and motor driver. There are 2 Hapkit-specific sensors connected to the Hapkit Board: Mr Sensor, and FSR Sensor. Thanks to the Hapkit Board’s standard shield shape and Grove interfaces, it is more convenient to connect manifold modules. For Hapkit Board, Seeed offered design for manufacturing services, customization services, marketing support, and sales of Hapkit Boards on our online shop.

Figure 4. Seeed Studio’s Hapkit Board

So, how does the Hapkit work (Figure 5)? When a learner touches the Hapkit with fingers and moves the paddle with force, the tactile data on motions and applied level of force will be recorded, integrated, and transferred from the Hapkit Board to the learner’s laptop. By creating various motions at different sequences, learners can experience different senses of programmed touch, be it, slippery, soft, hard, hot, cold, wobbly. For those who are interested in replicating Hapkit projects, you can buy our Hapkit Board here.

Figure 5. How Hapkit Works (Tedx Talks, 2013)

Through this Project, Dr. Okamura sent feedback to us that utilizing our Hapkit Board has resulted in dramatic cost reduction in building haptic devices compared to many previous models, as well as brought convenience for the learners. However, she demonstrates that the haptic devices are not just for teaching and learning, but can contribute to pragmatic solutions in our daily life. She says that, it is about controlling remote devices, so that one can replace the virtual world with a robot to tele-operate it. In this way, when the user moves, the teleoperated remote robot can follow their movements, and then, the user will feel what the remote robot is feeling (Figure 6). She further elaborates on how these devices can be used in real-life settings that need tele-operation, by taking examples of:

i) Robots dispatched in the universe;

ii) Demining robots for military and police to protect people from unpredictable harms when they have to take out explosives from the ground;

iii) Haptic devices differentiating different shapes and tactile feelings of tumors and cysts, and;

iv) Surgical robotics in the medical field used for going deep inside the body of a patient without making big holes in the body.

Unlike self-driving cars, these robots are not autonomous, but are teleoperated by humans at long-distance. With haptic devices, a surgeon can tie a suture with the most appropriate amount of force, so that the surgeon does not break the suture or the suture does not become loose. Without the haptic devices, the surgeons would not be able to figure out how much of the force they are applying to the suture and the patient’s body, when solely relying on visual screens.

Figure 6. Application of Haptic Device in Real-Life (Tedx Talks, 2013)

What’s more interesting is that, the educational haptics are recognized to transform our traditional education methods. For instance, different people learn in different ways: visual learners, auditory learners, and learners who learn by touching objects. For students who cannot grasp complicated mathematical or scientific equations easily, this will innovate how the students are learning – by translating those equations into something that the students can feel, thus, intuitively learn. In this way, we can advance the learning capacity of a huge number of people who would like to learn by facilitating their sense of touch (Figure 7).

Figure 7. Learning Equations By Using Haptics (Tedx Talks, 2013)

Which SDGs Are Relevant?

The potential of Hapkit, or open source haptic device, goes well beyond the realm of education, to various industries as entertainment, game, telecommunications, social welfare, and other imaginable and unimaginable scenarios that involve the sense of touch. As such, this Hapkit Project seeks to achieve the UN’s SDGs illustrated in Figure 8.

Figure 8. SDGs Connected to Hapkit Project  (UN, 2016)

To gain more insights on these these 3 SDGs, it is helpful to understand their specific Targets to realize why they are relevant with Hapkit Project:

1. Provide quality and affordable technical education at university level for all men and women to prepare them for decent jobs (Targets 4.3, 4.4)
2. Upgrade the usage of technology and ICTs to facilitate women’s empowerment (Target 5.B)
3. Provide mechanisms for capacity building on technology and innovation through multi-stakeholder partnerships among public and private sectors, as well as civil society organizations (Targets 17.8, 17.9, 17.16, 17.17)

All in all, today’s Hapkit Project showed us massive possibilities and significance of tactile technologies. As our world is increasingly becoming virtualized in digital space, Seeed provides customization project services from designing phase to production and marketing of the products, which are tailored to the client’s needs and vision. By collaborating with us, education experts and other industry players would be able to achieve digital transformation to be not left behind in this era of unpredictability. Join Seeed in the search for sustainable education for all. 🙂