Enginursday: A New Sensory Experience with the Cthulhu Shield

Many years ago I worked in the medical field helping those with physical and mental disabilities. Assistive/adaptive technology can be both prohibitively expensive and proprietary. This makes it hard to afford, and difficult to adapt to a particular need. When I heard that a local company was creating "the world's first open source sensory substitution/sensory augmentation development platform for Arduino," I grabbed my RedBoard Qwiic and eagerly started planning a project.

The Cthulhu Shield

For those unfamiliar with the Cthulhu Shield, it is an R3 footprint Arduino shield able to output to and take input from an electrode array that is placed on your tongue. This allows for an individual to control things like a mouse with only their tongue. It also allows for data output to the electrode array, where it produces sensation through biphasic voltage pulses on the electrode. The resulting sensation is akin to Pop Rocks candy. I wanted my project to be portable, so I powered my RedBoard with a simple 9V battery.

Cthulhu Shield

Retired SEN-15897

The Cthulhu Shield is the worlds first open source sensory substitution and sensory augmentation development platform.

Retired

Favorited Favorite 10

Wish List

If you are interested in the science behind it, or want to learn more about the company I recommend viewing their Kickstarter Page.

Sensory Substitution with Distance Sensing

I wanted to test out how well I could rely on a sensor in place of my eyes to navigate the building. I used our VL53l1x Distance Sensor and set three different thresholds. If a detected object was under 3 feet away, the electrodes would be set to an intense output; between 3-10 feet, a medium output; and no output if the object is over 10 feet away.

#include <Cthulhu.h>

#include <ComponentObject.h>
#include <RangeSensor.h>
#include <SparkFun_VL53L1X.h>
#include <vl53l1x_class.h>
#include <vl53l1_error_codes.h>

#include <Wire.h>
#include "SparkFun_VL53L1X.h"



int started = 0;



SFEVL53L1X distanceSensor;
//Uncomment the following line to use the optional shutdown and interrupt pins.
//SFEVL53L1X distanceSensor(Wire, SHUTDOWN_PIN, INTERRUPT_PIN);


Cthulhu mycthulhu; //creating and instance of Cthulhu



int array[18];

//array to hold which electrodes should be on or off.
int trodes[] = {1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};

//pulse period for each electrode, in microseconds. Can be manipulated with Pp and IN to change the intensity of the sensation.
int  PP[] = {10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10};

//length of positive pulse for each electrode, in microseconds. Can be manipulated with PP and IN to change the intensity of the sensation.
int  Pp[] = {9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9};

//inner burst number (how many pulses in each inner burst). Can be manipulated with PP and Pp to change the intensity of the sensation.
int  IN[] = {3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3};

//inner burst period.  In microseconds. Can change quality, or 'feel' of sensation.
int  IP[] = {150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150};

//Outer burst number. Can change quality, or 'feel' of sensation.
int  ON[] = {5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5};


//Requirements:
//Pp must be less than PP.                    CheckWaveform Error 1.
//(PP*IN) must be less than IP.               CheckWaveform Error 2.
//IP*IN must be less than 2000 microseconds.  CheckWaveform Error 3.


void setup(void)
{
  Serial.begin(9600);

}

void loop(void)
{
  Wire.begin();

  mycthulhu.Begin(); //Initialize Cthulhu library
  oldone();

  distance();
}





void oldone()
{



  //mycthulhu.Begin(); //Initialize Cthulhu library
  //check our waveform parameters and verify that they meet requirements
  int myerror = mycthulhu.CheckWaveform(trodes, PP, Pp, IN, IP, ON);

  if (myerror == 0) //if our waveform parameters are ok, update the stimulation parameters and perform one 36ms stimulation cycle.
  {
    mycthulhu.UpdateStimuli(trodes, PP, Pp, IN, IP, ON); //update waveform parameters
    mycthulhu.Stimulate(); //create the stimulation pulsetrain on the electrodes
  }
  else
  {
    //if we entered an invalid waveform, tell us which rule we violated so we can fix it
    Serial.print("Error in waveform parameters. CheckWaveform Error: "); Serial.println(myerror);
  }
  // Wire.end();
  delay(100); // increased delay to avoid eagleye conflict.
}

void distance ()
{




  distanceSensor.begin();



  distanceSensor.startRanging(); //Write configuration bytes to initiate measurement
  int distance = distanceSensor.getDistance(); //Get the result of the measurement from the sensor
  distanceSensor.stopRanging();

  Serial.print("Distance(mm): ");
  Serial.print(distance);

  float distanceInches = distance * 0.0393701;
  float distanceFeet = distanceInches / 12.0;

  Serial.print("\tDistance(ft): ");
  Serial.print(distanceFeet, 2);

  Serial.println();
  if ( distanceFeet < 3)
  {
    for (int i = 0; i < 18; i++)
    {
      trodes[i] = 1;
      PP[i] = 40;
      Pp[i] = 39;
      ON[i] = 8;

    }
  }

  if (( distanceFeet < 10) && (distanceFeet > 3))
  {
    for (int i = 0; i < 18; i++)
    {

      trodes[i] = 1;
      PP[i] = 10;
      Pp[i] = 9;
      ON[i] = 5;

    }
  }
  if ( distanceFeet > 10)
  {

    for (int i = 0; i < 18; i++)
    {
      trodes[i] = 0;
      PP[i] = 10;
      Pp[i] = 9;
      ON[i] = 5;

    }
  }

}

added to your cart!

SparkFun Distance Sensor Breakout - 4 Meter, VL53L1X (Qwiic)

In stock SEN-14722

This SparkFun Distance Sensor Breakout utilizes the VL53L1X next generation ToF sensor module to give you the highly accurate…

$23.50

10

Favorited Favorite 52

Wish List

Sensory Augmentation with Eagle Eye IR sensor

When I was thinking of what new sense I wanted to have, I immediately thought of a certain alien hunter - a predator, if you will - that used infrared vision to track its prey. Snakes also have a pit organ that allows them to use IR.

I used our Eagle Eye IR sensor, which conveniently already uses a grid array, and I reduced the resolution from 64 points to 16 for the Cthulhu Shield. This allowed for some positional sensation. If a "hot" object approached from my left into the field of view I would first feel it on the left side of my tongue. As the object moved through my FOV, or as I moved my sensor toward the object, I could feel when it was centered in front of the sensor.

Naturally, I attached the sensor to a mask, placed the electrodes in my mouth, grabbed a Nerf gun and started the hunt.

#include <Cthulhu.h>
#include <Wire.h>
int started = 0;

#include <SparkFun_GridEYE_Arduino_Library.h>


#define HOT 30
#define COLD 20

// This table can be of type int because we map the pixel
// temperature to 0-3. Temperatures are reported by the
// library as floats
int pixelTable[64];
//int transarray[64];
GridEYE grideye;



Cthulhu mycthulhu; //creating and instance of Cthulhu



int array[18];


//two arrays I used to generate geometric patterns

int cElectrodeMap[5][4] =
{
  {0, 0, 1, 0},
  {2, 3, 4, 5},
  {6, 7, 8, 9},
  {10, 11, 12, 13},
  {14, 15, 16, 17}
};

int on[5][4] =
{
  {0, 0, 0, 0},
  {0, 0, 0, 0},
  {0, 0, 0, 0},
  {0, 0, 0, 0},
  {0, 0, 0, 0}
};


//array to hold which electrodes should be on or off.
int trodes[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};

//pulse period for each electrode, in microseconds. Can be manipulated with Pp and IN to change the intensity of the sensation.
int  PP[] = {40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40};

//length of positive pulse for each electrode, in microseconds. Can be manipulated with PP and IN to change the intensity of the sensation.
int  Pp[] = {39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39};

//inner burst number (how many pulses in each inner burst). Can be manipulated with PP and Pp to change the intensity of the sensation.
int  IN[] = {3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3};

//inner burst period.  In microseconds. Can change quality, or 'feel' of sensation.
int  IP[] = {150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150};

//Outer burst number. Can change quality, or 'feel' of sensation.
int  ON[] = {8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8};


//Requirements:
//Pp must be less than PP.                    CheckWaveform Error 1.
//(PP*IN) must be less than IP.               CheckWaveform Error 2.
//IP*IN must be less than 2000 microseconds.  CheckWaveform Error 3.

void setup(void)
{

}

void loop(void)
{


  mycthulhu.Begin(); //Initialize Cthulhu library
  oldone();

  Wire.begin();


  Serial.begin(115200);
  eagle();
}

void oldone()
{



  //mycthulhu.Begin(); //Initialize Cthulhu library
  //check our waveform parameters and verify that they meet requirements
  int myerror = mycthulhu.CheckWaveform(trodes, PP, Pp, IN, IP, ON);

  if (myerror == 0) //if our waveform parameters are ok, update the stimulation parameters and perform one 36ms stimulation cycle.
  {
    mycthulhu.UpdateStimuli(trodes, PP, Pp, IN, IP, ON); //update waveform parameters
    mycthulhu.Stimulate(); //create the stimulation pulsetrain on the electrodes
  }
  else
  {
    //if we entered an invalid waveform, tell us which rule we violated so we can fix it
    Serial.print("Error in waveform parameters. CheckWaveform Error: "); Serial.println(myerror);
  }
  //Wire.end();
  delay(100); // increased delay to avoid eagleye conflict.
}

void eagle ()
{
  int x = 0;
  //Wire.begin();
  grideye.begin();

  // loop through all 64 pixels on the device and map each float value to a number
  // between 0 and 3 using the HOT and COLD values we set at the top of the sketch
  for (unsigned char i = 0; i < 64; i++) {
    pixelTable[i] = map(grideye.getPixelTemperature(i), COLD, HOT, 0, 3);
  }


  // loop through the table of mapped values and print a character corresponding to each
  // pixel's temperature. Add a space between each. Start a new line every 8 in order to
  // create an 8x8 grid
  for (unsigned char i = 1; i < 64; i++) {
    if (pixelTable[i] == 0) {
      Serial.print(".");
      if ( (i % 4 ) == 0)
      {
        trodes[i / 4] = 0;
      }
    }

    else if (pixelTable[i] == 1) {
      Serial.print("o");
      if ( (i % 4 ) == 0)
      {
        trodes[i / 4] = 1;
      }
    }
    else if (pixelTable[i] == 2) {
      Serial.print("0");
      if ( (i % 4 ) == 0)
      {
        trodes[i / 4] = 1;
      }
    }
    else if (pixelTable[i] == 3) {
      Serial.print("O");
      if ( (i % 4 ) == 0)
      {
        trodes[i / 4] = 1;
      }
    }
    Serial.print(" ");
    if ((i + 1) % 8 == 0) {
      Serial.println();
    }
  }

  // in between updates, throw a few linefeeds to visually separate the grids. If you're using
  // a serial terminal outside the Arduino IDE, you can replace these linefeeds with a clearscreen
  // command
  Serial.println();
  Serial.println();


  // toss in a delay because we don't need to run all out
  delay(100);


}

added to your cart!

SparkFun Grid-EYE Infrared Array Breakout - AMG8833 (Qwiic)

Out of stock SEN-14607

The SparkFun Grid-EYE Infrared Array Breakout board is an 8x8 thermopile array, giving you a square of 64 pixels capable of i…

$45.95

2

Favorited Favorite 39

Wish List

Moving Forward with the Cthulhu Shield

It takes time and practice to start building sensitivity to the position and strength of the electrodes on the array. I changed the output settings on the electrodes to produce as much sensation as I could to help with my uninitiated tongue. I plan on reducing this as I get better at parsing out strength and position of the electrodes. I also experienced an issue when using I2C devices where if I completely bit down on the electrode it would cause the board to freeze where it was in the loop. As a result I needed to gingerly place it on my tongue or block one of the electrodes with some plastic to prevent the freeze up. I am not sure why this occurred, but that is what oscilloscopes and GitHub issue requests are for!

With a whole world of sensors and data out there, the limit to the Cthulhu Shield is our imagination. If you could give yourself a new sense what would it be? The ability to sense total Volatile Organic Compounds? Use GPS to help guide yourself to your destination?