Open Source Ethics for Autonomous Surgical Robots
4 days workshop at Kyoto Institute for Technology (D.Lab) on the future of elder care.
Project developed with Frank Kolkman, Henrik Nieratschker and Jaime Garcia

We looked at how DIY could contribute to the future of eldercare. Indeed, with issues such as high expenses, and the fact that the elder frequently has to leave their house to go to a nursing home, a DIY eldercare movement could offer more pleasant ways to live out one’s senior years. Not only would expenses be decreased – as one of the concepts of this culture is to not rely on expensive paid experts – but also, more importantly, the elder is not bound anymore to any kind of structure. They can arrange their DIY technology to the way they feel it will best accommodate their lives.

My interest for this project was not focused on the DIY device but on the DIY culture. Undeniably, like the open-source movement in software engineering, such cultures are not only about building cheap/free devices but also about developing a strong community composed of individuals who commit their spare time to build, share and discuss artefacts that matter to their beliefs. Most common DIY or open-source websites, such as Instructables or GitHub, are heavily structured to facilitate this human exchange.

Thus our proposal was about a DIY autonomous robotic surgical system. For this workshop, I was focusing on the AI side of this robotic system as we speculated it would be autonomous, while the other team members focused on the product in itself (the DIY side) - please see Frank Kolkman’s project which is an excellent follow up!

I looked at one routine/scenario that this robot might encounter: how should it act if it suddenly detects that the patient is losing blood during a surgical operation?

The main algorithm was operating in the following way:

1. Assessing the seriousness of the bleeding rate; it could be low, medium or high.
2. Depending on how serious the bleeding rate is, it applies one of these four techniques: cauterization, medication (blood coagulant), blood salvage and, finally, blood transfusion. These techniques are from the least (cauterization) to the most (blood transfusion) dangerous for the patient’s health. However, the least dangerous is also considered the least effective.
For instance, a low bleeding rate would make the robot starts a cauterization, whilst the high bleeding rate would trigger a blood salvage.
3. If the robot fails to fix the bleeding, it would apply the next technique, e.g. if it fails with cauterization, it moves to the blood coagulant phase, and so on.
4. The robot resumes the surgery only once the blood loss has been successfully fixed.

This is how the main software is behaving. If a company manufactured this robot, this software it would be “closed” and would always operate in the same way, no matter who the patient is. However, it is not the case here as this robot is open-source. The software is “open” and anyone could alter its behaviour by modifying its various routines. As fixing a blood loss can quickly become a controversial operation, a machine operating on such scenario in an autonomous way should have some kind of ethical awareness. So we could imagine that different communities – defined by their religion in our example – would change how the robot is dealing with that blood loss.

For instance, Jehova Witnesses refuse blood transfusion. Thus a group of Jehova Witness hackers develop a module on top of the main software in order to forbid the robot to perform a blood transfusion. As an alternative, they develop a module that can surgically close the vein of the patient (a very dangerous operation seldom used, but the only alternative to a blood transfusion). Another example is given regarding a group of Jewish people who refuse to take medication that originated from Nazi experiments – which is the case of blood coagulant medicine – thus this community develops another module for the main software that avoids the blood coagulant phase and jumps instead to the next step.

Due to the very nature of open-source: its strong sense of community and the fact that the software is open and thus alterable; an open-source autonomous system is a concrete example of how ethical subjectivity could be more or less implemented into one product.