For this week’s assignment, we had to make a web dashboard for controlling a Philips Hue bulb. I was pretty stretched for time this week so I decided to keep things simple and learn the basics. Going through the tutorial was pretty self-explanatory and the code on github was pretty logical. However, I tripped up with the call-backs and it was quite confusing for me to understand. Thankfully, I read through Timothy and Atharva’s blog and their call-backs made sense to me. I created a basic UI where the Hue can be controlled by changing it’s Hue, Saturation and Brightness values through sliders and also gave the users the option to turn it on and off. I was also changing the background of the webpage to match the color of the Hue bulb.

Screenshot 2019-02-12 04.26.39.png

I was looking at changing the color of the text to a complementary color based on the currently selected value but did not find any easy way to convert HSB values to its complementary colors. Also, I had thought of an ambient mode where the digits of HH:MM:SS are converted into a RGB hex value which is then transmitted to the hue. Again, I tripped up because I could not find a reliable way to do this. Passing RGB to HSL did not match the colors. So my top questions for this week would be:

1) How do you calculate the complementary HSB value of a color through code?

2) How do you convert a rgb color to a HSB color?

Currently listening: A whiter shade of pale - Procol Harum

References used:


//IP address of Philips Hue
let IPHub = '';
let userName = 'g12MalhwYs2DUmfQhW-GYJEzwYsV3mYxiKJLXuxb'; // My user name as per the hue developer API
let url;

let gotham;

//Variables for display of controls and labels
let checkBox;
let hueSlide;
let hueText;
let satSlide;
let satText;
let brightSlide;
let brightText;

//Variables for controlling the philips bulb and its color
let lightNum = 1;
let hueVal = 32767;
let satVal = 127;
let brightVal = 127;

//Loading fonts
function preload() {
  gotham = loadFont('assets/Gotham Book.otf');

function setup() {

  //Create canvas
  canvas = createCanvas(windowWidth, windowHeight);
  canvas.background(hueVal, satVal, brightVal);

  //Declare Hue URL
  url = "http://" + IPHub + "/api" + userName;

  //Position ON/OFF checkbox
  checkBox = createCheckbox(' IS ON/OFF', false);
  checkBox.position(canvas.width / 2 - 90, 200);

  //Position Sliders for color control
  hueSlide = createSlider(0, 65535, 32767, 100);
  hueSlide.position(canvas.width / 2 - 225, 400);'rotate', 90);

  satSlide = createSlider(0, 254, 127, 1);
  satSlide.position(canvas.width / 2 - 100, 400);'rotate', 90);

  brightSlide = createSlider(1, 255, 127, 1);
  brightSlide.position(canvas.width / 2 + 25, 400);'rotate', 90);

  //Position button for 'Ambient Mode'
  button = createButton('Ambient mode');
  button.position(canvas.width / 2 - 85, 600);


function draw() {


  if (hueSlide.value() != hueVal || satSlide.value() != satVal || brightSlide.value() != brightVal) {

    //Change color

    //Capture slider value
    hueVal = hueSlide.value();
    satVal = satSlide.value();
    brightVal = brightSlide.value();

    //Change background
    colorMode(HSB, 65535, 254, 255);
    background(65535 - hueVal, 254 - satVal, 255 - brightVal);

    //Display text
    textSize(width / 15);
    textAlign(CENTER, CENTER);
    text('Huehuehue', width / 2, 100);

    hueText = textSize(width / 60);
    hueText.text('Hue', canvas.width / 2 - 160, 500);

    satText = textSize(width / 60);
    satText.text('Sat', canvas.width / 2 - 35, 500);

    brightText = textSize(width / 60);
    brightText.text('Brightness', canvas.width / 2 + 100, 500);


function toggleLight() {

  let path = url + '/lights'
  httpDo(path, 'GET', toggleGetResponse);


function toggleGetResponse(getData) {

  let lights = JSON.parse(getData);
  lightState = lights["1"].state.on

  let body = {
    'on': !lightState
  let path = url + '/lights/' + lightNum + '/state/'
  httpDo(path, 'PUT', body, togglePutData);


function togglePutData(putData) {

  var response = JSON.stringify(putData);
  if (response.includes("success")) {
    lightState = !lightState

function changeLightColour() {
  var path = url + '/lights/' + lightNum + '/state';
  var body = {
    'bri': 255 - brightSlide.value(),
    'sat': 254 - satSlide.value(),
    'hue': 65535 - hueSlide.value()
  var path = url + '/lights/' + lightNum + '/state/'
  httpDo(path, 'PUT', body, changeColourResponse);

function changeColourResponse() {
  console.log('Colors changed!');

function changeBG() {
  let hr = hour();
  let mn = minute();
  let sc = second();


Disobedient electronics

The theme for our second assignment was to create an object that exemplifies the ethos of disobedient electronics. I teamed up with Winnie Yoe and in our first discussion, we decided to make a few learning objectives for ourselves. Our initial list was: 1) Learn how to use ESP32 2) Learn how to fetch and display real time data 3) Use data to work with a mundane regular object that we see day to day.

Initially, we looked at the NYC open data sets and we found some interesting data around maternal health, mental health and the drug crisis. We were interested to use the data set for the opioid crisis but we realised that none of the datasets we had was not real-time and had no granularity beyond a district zone. Working with such large data-sets was proving to be challenging and we gave up on the approach.

During the discussion, we started talking about how mundane objects are basically fronts for corporations inside our homes in the name of ‘Smartness‘. That struck a chord and we refined the idea into a simple ‘smart‘bulb that is free to use but it won’t light up if the latest stock market price of the company was lower than the previous day. Going through stock market prices api, we found one which was easy to use but only gave daily stock prices. We wanted one which was hourly but in the interest of time, we went ahead with the one we found to build the proof of concept. We used the ESP32 HUZZAH to control the light bulb.

The final interaction was as follows: The bulb lights up if it detects the presence of the user and then checks for the stock price of the company (*cough* Facebook *cough*) and if the price of the company was lower, it starts blinking annoyingly. The user then has to mash the ‘like‘ button which leaves gratuitous comments on social media (not prototyped) and the bulb is ready for use again. You can watch the interaction in the video below.

I was quite happy about getting the APIs to work with the chip. I realise that there are conceptual gaps in our prototype but a lot of it was pared down in the interest of time. I believe that there is enough depth in the concept to take it further and I would like to see if I can do the same project in a more refined manner later.

A piece of velvet

I grew up in a small city in India. It was hot, dry and utterly boring in a way that only small cities in 80’s India can be. I was born premature which made me pretty sick through my early years and having no brothers and sisters, I was pretty much in my own head. And a bursting imagination often needs outlets and for me, in came in the form of playing with wooden toys. My family did not have a lot of money, so LEGOs, action figures and toys were out. But as a child, who cared? A few blocks of wood, plastic and boxes and you got a castle going! And in 80’s India, no one around me had any expensive, manufactured toys. So, it wasn’t as if I felt the need to have something that wasn’t being given to me. I was pretty happy in my own head until I saw an advertisement for a GI Joe.


GI Joes were probably the first thing I ever wanted. I was entranced and I remember throwing tantrums for having them. My parents couldn’t really afford them so they would try to keep me away but being a male child in India comes with doting grandparents and uncles who would try and cater to my whims. My frustrated parents couldn’t really say anything and in a few years, I had a collection of about 50 of them.


But this is not the story of the GI Joes.


One summer, while spending the summer vacation at my maternal grandmothers place, I came upon a box which had a small piece of velvet, 2 tiny pillows and a small piece of wood. The velvet was bedraggled with aluminum milk bottle caps stuck on it, the pillows were made out of cloth and the piece of wood, was well, a piece of wood. When I asked my grandmother about it, she told me about how that was a bed for my mother’s doll. My family is one that was torn apart from the partition of India and both my grandfathers had to leave everything they knew behind to start from scratch. So, we never had a lot of money and buying a doll was impossible. But that didn’t stop my grandmother and mother. They made dolls from whatever material they could find, built a bed, a blanket and pillows. My mom grew up playing with a stuffed piece of cloth and treasured it long after she had outgrown them. On that summer afternoon, it all came rushing to me about how entitled was I to ask for an expensive piece of plastic which was way above our means but my parents still tried to do the best they could. I felt the insides of my stomach churn and I had no way to understand what I was feeling as a child but that feeling created a sense of gratitude for them trying to do the best for me in whatever way they could. The little piece of velvet became a part of my GI Joe collection. After a long, hard day of fighting, they all were put to sleep under my mom’s velvet blanket. After all, warriors need to sleep. I often wonder what they dreamt of? What would people who fought all day dream of? Do they dream of peaceful times or more war? And under the glittering, shiny blanket, would they be happy? I did not know but it was fun to imagine that.


GI Joes unleashed my imagination. Simulation video games and construction kits later shaped my intellect, thinking and unleashed my ability to make. But my mother’s piece of velvet taught my gratitude, kindness and softness. And for that, I am grateful. Growing up as a man in India, you have a lot of hard edges as a patriarchal, masculine society shapes you to be. But a piece of cloth can round you and round you out like stones in a river. Who could have guessed?

Week 1: Just another basic server.

For this week’s assignment, we were asked to create a simple http server using node.js and express. Both these terms were completely new to me and I decided to keep my ambitions in check and build something that works instead of the glorious failures of ICM and P.Comp in the semester past.

For starters, I familiarized myself with node and express with Dan Shiffman’s videos. (Link)

The ‘Programming A to Z’ website also has some great explanations on working with node.js and express (Link)

For my assignment, a combination of hanging out with small bots in ‘Hacking smart toys for AI learning‘ and listening to Leonard Nimoy narrating Ray Bradbury’s ‘There will come soft rains‘, I decided to make a web server to control a bot in the following ways:

  • Make the robot move ahead. (/forward)

  • Make the robot move behind. (/back)

  • Make the robot turn left or right. (/turn/[:left or :right])

  • Make the robot dance. (/happydance)

The code was pretty uneventful except the part of constantly having to turn the server on and off. Another part which tripped me over was that home I was encountering an error when I was trying “My network IP“ :8080 instead of localhost:8080. It works like a charm inside ITP though. Maybe, it was happening because I was on a hotspot but I had no idea to rectify it. I would like to know more about how to identify and rectify such network issues.

/* References used4-line server example from Tom Igoe's class: Shiffman's videos from coding train:*/

//Include express

let express = require('express');

//Create a server

let server = express();

//Serve static files from public

server.use('/', express.static('pages'));

//GET parameters

server.get('/turn/:direction', turnBabyTurn);

server.get('/forward', moveForward);

server.get('/back', moveBack);

server.get('/happydance', happyDance);

//Start the server


//Functions to send response to GET requests

//robot turn

function turnBabyTurn(request, response) {    

let newTurnState = request.params.direction;    

if (newTurnState == 'left') {        

response.send('The robot makes a sharp turn to the ' + newTurnState);} 

else if (newTurnState == 'right') {        

response.send('The robot makes a sharp turn to the ' + newTurnState);} 

else {       

 response.send('Something went terribly wrong. Bots are stupid like that. try left or right?');}    

response.end();}//Move back

function moveBack(request, response) {    

response.send('The bot retreats back not knowing what’s lies behind it.');    


//Move forward

function moveForward(request, response) {   

response.send('The bot whirrs forward towards an indeterminate future.');    



function happyDance(request, response) {    

response.send('The bot spins on its own axis silent and alone.');    


Currently listening: Keep Talking-Pink Floyd

Apology as a Service (AAAS)

For our first assignment for critical objects, I teamed up Winnie Yoe to work on a critical object. The shop was shut for the week and we started talking about how we should write an apology for not doing the assignment. This led us to a further discussion of how apologies are manufactured and it’s as if they are a formula.

We went down the rabbit-hole of digging up apologies from Kevin Spacey to facebook to Uber and many others and we came up with the formula as follows:

[Inspirational title] → [Demonstrate passion] → [Play the Victim] → [Feign innocence of events] → [Cautiously appreciate the victims] → [Ask for time]→ [Recognise role of company without any direct acceptance of wrong-doing]→ [Promise indeterminate actions in the future]→ [Promise that it won’t happen again]→ [salutations]→ [Actual signature].

While analysing the responses, we are realised that the formula also caters to institutional anxieties and are about protecting the organisation rather than the aggrieved ones. We came up with the idea of a service for CEOs in the future which is a voice driven interface for generating apologies. We name it Bernays after Edward Bernays, the father of modern PR quite extensively documented in The century of the self.

The hypothetical device sits on the desk of the CEO and talks to him/her about the current issue and uses advanced AI to understand the situation. It asks the CEO for ‘uncomputable‘ information which helps it create a more nuanced approach to a situation and generates an apology and a strategy for handling the situation.

You can scroll through the UI below. A sister post on the project can be found here.

Animation: Unity final

I have always wanted to build an underwater escape room experience and the unity assignment was the perfect foil for doing the same. When I started on the assignment, I knew that I wanted the following things:

  • Build a terrain of my own in Unity.

  • Play with lighting of the environment.

  • Create a responsive environment that acknowledges your presence.

  • Use the unity animator tool.

While my initial plan was to build an experience in which the person is trapped inside a ship with a shark trying to break in from the outside, it was out of my technical scope. So, I pared it down to something manageable that I could do with the finals madness going on.

Making the terrain in unity was pretty trivial but the models proved to be a huge pain. Initially, I used extra detailed models that I found on thingiverse and other places but they kept dropping the frame rate. So, I had to keep paring down my polygon models to make them work.

The schools of fish also proved to be a big problem. I had 7 schools of fish in the environment but they dragged down the frame rate. So, I had to write a script that they will be triggered only when I am near their start location. Then, I wrote one more script which makes them hurry away from you in a random direction when they come near you. Once the fishes were done, I focused on the sharks.

I created 2 parts of the shark. One was the body and the other was the mouth. The mouth was animated in unity to snap back and forth as it moves towards the player when one ventures into their area. (Basically, a big hit box triggers the shark to come towards you).

You can see the piece here:

FAB 6: Mounting motors.

The last assignment. And probably my worst. What I wanted to build was a box with a button and a title saying “How will your day at ITP be today?” and when you press the button, the windmill moves to a random answer.


However, with the final week madness and my P.Comp project being a tangle of wires and a mess of code, I didn’t have the mindspace or energy to make it. So, I probably pulled off the worst job of all time and used hot-glue, reclaimed wood and mounted the motors with basic screws to create something that works.

I did not get time to either program the random behavior or do anything else with it. The only thing that I did in this that I was remotely proud of was to turn the circular piece of wood. I call this piece “Turning it in“.

FAB 5: Two Materials.

For this assignment, I had to use two materials that work together. I have always to build a lamp which is part silicon and wood. However, I have been working with silicon a lot lately and I decided to go with epoxy so that I could try out a new material for a change.

I wanted to interplay between the relative hardness of the 2 materials. Epoxy is fluid while wood is hard and I wanted to make something which reflects that in its form. After sketching multiple ideas, I honed in on this one.


I wanted to build a lamp which had wooden supports but had epoxy in the middle which lights up. Once I had a direction in mind, I got started on the wood and making the base. Wood from the junk shelf, an angled cut from the miter saw and finishing on the sander gave me 2 pieces which were perfectly matched in form.

Using plain ol’ geometry to line up the holes.

Using plain ol’ geometry to line up the holes.

The next step was to make holes. I made a through hole in the center for the LEDs to pass through and two small holes for the epoxy to flow and harden so that it doesn’t slip. (Remember Mark, you said no screws!) Once I had that, I tested an LED strip and pared it down to size.


The next step was to draw an outline and start forming the mould.

Lining the paper with plastic sheet to prevent leaks.

Lining the paper with plastic sheet to prevent leaks.

Forming the first wall.

Forming the first wall.

Filling up the wooden blocks with plasticine clay to prevent leakage.

Filling up the wooden blocks with plasticine clay to prevent leakage.

Forming the outer layers and using clay to position them.

Forming the outer layers and using clay to position them.

Clamping it down.

Clamping it down.

Using hot glue and plasticine to fill up blank spaces and create a water resistant mold.

Using hot glue and plasticine to fill up blank spaces and create a water resistant mold.

Getting the poxy ready. I wanted to do multiple colors to create a marbled effect.

Getting the poxy ready. I wanted to do multiple colors to create a marbled effect.

So, one would expect that it would go smoothly right? I had the mould ready, the epoxy in a glass and it was all looking good.

But I made 1 big mistake. The epoxy I chose doesn’t play well with the plasticine I used. The epoxy also undergoes an exothermic reaction which basically made the plasticine more sticky and ensured that my structure failed on me. I had taken a small nap after pouring the epoxy and woke up to the epoxy fluid seeping all over the floor in the shop and creating a huge mess everywhere. Thankfully, there weren’t any people around as it was 6 in the morning and I scrambled to clean it before John came in. I finished all the tissues in the kitchen and the loos to mop that mess up and had to walk back home smelling of epoxy fluid and defeat.

While the end of the process was a complete disaster, It was education in itself. I hope to repeat this all over again soon and make something that really works and sets.

FAB 4: Enclosure madness.

So, this week out mission was to build an enclosure. (Rubbing hands in glee)

WARNING: Long blog-post ahead.

TL,DR: I build a box for my physical computing project. It was very pretty.

This assignment segued neatly into my physical computing project. for more on the project and it’s background, please go here.

The first sketch of the box was this:


Initially, I did not have any idea of the dimension or the scale of the box. So, the first stage was to finalize the puzzles and the ergonomic size which was in line with the theme. So, I did just that and finalized the puzzles and the tentative layout.


I knew that I wanted the box to be big but still fit within people’s hands. A quick test with people on different sizes on the floor and I fixed on it being about 3 feet wide. I also wanted it to resemble the control panels of old on the outside. The insides needed more careful consideration. The box had to be sturdy enough to handle people playing with it so it needed enough cross-bracing. Also, the top panel needed to be swapped in and out so that the box layout could be iterated upon and also adjust the circular screen in the center. Once the dimensions were fixed, the box was built in 2 phases:

Phase 1: Acquire the buttons, knobs and dials and figure out their mounting. This part was probably my favorite in the semester. I spent a good part of 3 days deciding on the buttons, knobs and dials and trawling through the depths of the internet in acquiring them.

Once they arrived, I spent a day playing with them and Lillian spent the time measuring every small detail of them with the callipers. Bringing them over to illustrator and we were ready to do the mounting tests. At this point, it was done on both acrylic and wood so that there was enough flexibility in the future regarding the choice of materials.

Once they arrived, I spent a day playing with them and Lillian spent the time measuring every small detail of them with the callipers. Bringing them over to illustrator and we were ready to do the mounting tests. At this point, it was done on both acrylic and wood so that there was enough flexibility in the future regarding the choice of materials.


Phase 2: Once the dimensions were figure out, the next task was to build the box. Initially, the idea was to use hardwood but we fell back on ply as it was easy to obtain in the dimensions we needed (and cheaper too). A combination of the miter saw, band saw, sander and voila!


Once the box frame was done, we spent a week or two play testing and getting the layout right. A week before the submissions, we started the final mounts.


All ready for the show! Let’s see how this goes <3


14/14 Posenet rabbithole

As mentioned previously, my ICM finals was to design a Pose-Karaoke experience. For my motivations and background on the project, please go to the post here.

While I had major issues with getting the ICM code to work on the ml5js platform, much of it has been rectified by the maintainers of the code-base and this current example solves pretty much all the problems.

But while I was doing this, I did not have those luxuries. This resulted me in understanding how Javascript works and figuring out the issues myself. The main issue was that a P5Image does not have a img html tag that ml5js needs to be able to run the algorithm. (Funnily, it works for video. No clue why this is done this way). This was solved using an image tag. But the problem with taking a screenshot from the live video still remained. I soldiered on and found my redemption in the toDataUrl() method.

But this was the easy part.

While starting the project, I did not realise the complexity of comparing 2 poses. A lot of what I had to do relied on being able to compare 2 images and it wasn’t a trivial problem. Trawling through the depths of the internet, I came across this post by Google research where they had worked on a similar problem. This post is a wealth of information on how to compare poses and it was outside my technical ability to be able to incorporate everything in my work. But the chief things that I could incorporate were:

1) Cosine similarity: It is a measure of similarity between two vectors: basically, it measures the angle between them and returns -1 if they’re exactly opposite, 1 if they’re exactly the same. Importantly, it’s a measure of orientation and not magnitude.

2) L2 normalization: which just means we’re scaling the vector to have a unit norm. This helps in ensuring that the scale does not play a factor in comparison and the 2 images can be compared normally.

The cosine similarity helped my code run faster and the L2 normalization ensured that the relative distance from the camera won’t play a role in the comparison.

Getting these 2 things to work proved to be a big challenge and once that was done, the comparison went pretty smoothly as seen in the video below:

I ran out of time to build a complete experience for the users which involve an engaging UI but that gives me something to do for the winter break. While I could not match the scope I had set initially, I am very happy that I could dive into algorithmic complexities and solve those issues to make something working. This gives me a lot of hope for the future and my coding abilities. All in all, time well spent!

Understanding comics

For animation class, we were asked to read ‘Understanding comics‘ and reflect on what we have learnt. I have read the book many years ago and it was great to pick it up and go through it all over again after so many years. I had originally read the book before I started design school and I realised that I forgotten so many aspects of the book which emerged to me the second time around. Here is a list of my reflections that I noticed and picked up on my 2nd read-through:

  • The narrative is more immediate as compared to film. While we demand narrative coherence in film as we respond to the flow of ‘time’, a comic is free because it can move through time and space in a matter of few panels. I hypothesize that its one of the reasons why comic book plots don’t translate well on screen where the audience responds more to the flow of events across time rather than the space of a comic book.

  • The role of a narrator: I believe that the narration is the anchor which hold the comics together. Which is why I haven’t seen many works where the narration and visuals are at odds with each other. It will be interesting to see a narrative where the visuals and the narration start diverging and running at total odds with each other.

  • Panel to panel transitions: This was the biggest part of the book that I had totally forgotten. Scott Mccloud does a great job at explaining the various ways in which a narrative can work across time and space using the 2 dimensional paper grid. This got me thinking about the forms that I see back home. Is there an inherent structure to how a story manifests in a mandala or on the wall of an Indian temple? Do similar rules apply> I believe that there should be one but hopefully, I will find a book that talks about it in detail.

One of the biggest things that struck me while reading this book was that the constraint of the medium squeezes out the narrative style and structure. While the boxes might be seen as constraints by some, artists used it to tell their stories in unique ways that have now become representative of the medium. I wonder if there is a similar story with VR. While VR does not have any control over the user’s view-point, what are it’s unique constraints from which VR-only narratives will emerge?

After effects project proposal.

For the after-effects project, I am teaming up with the mighty Cara. During our first brainstorm, I fell in love with the spaceman character that Cara had come up with and we decided to use the character in a museum dedicated to Earth after it has been destroyed.

The plot:

An anonymous astronaut visits the Museum of Earth, the last surviving vestige of the planet destroyed in the water wars of 2050. The screens consist of short montages of what it means to be human. The animation cycles through 4-5 animations.


Pretty excited to see where this goes. After effects is daunting AF but we can power through this.

FAB 3: Shattered, does it matter?

This week’s assignment was to make something out of acrylic using the laser cutter.

Easy peasy. I have been meaning to do a Voronoi lamp for the longest time and finally, it’s time to scratch that itch. So, off to p5 I went. Now, things are much easier compared to the last time I worked with voronois and guess what? there are libraries for that now.

After playing around with the shapes and size, I brought a few patterns that I liked into Illustrator. I played around with the dimensions of my box and tried to find a cross-section that won’t have very tiny shapes that might mess up with the laser cutter.

Screenshot 2018-11-16 14.41.39.png

Once I had that, it was time to find the material. Now, my original plan was to have 6-10 colors but looking at the costs and the availability of plastics, I brought it down to 4.

Screenshot 2018-11-16 14.46.44.png

I separated the colors into individual files for each color and off to the laser printer I went. Cutting the pieces was pretty uneventful except the part where I lost a nice slab of acrylic to the 75w printer which refused to work. (The cutting gods always demand a sacrifice) and within 45 minutes I had all my pieces. (easy peasy!)


Sticking the acrylic was a different monster altogether. The adhesive that I had was so runny that it was making my life miserable. But thankfully, Lydia got me out of a soup and loaned her rubber cement to me which made my life so much easier. And voila, within 2 hours I had a lamp!


All I need to do is find some LEDs to light it up and it shall be AMAZING!.

Finals madness!

For my final in physical computing, my initial direction was to continue working with soft robotics. I wanted to explore the material more and create data-driven experiences that with softness, slowness and reflection as guiding principles. I brainstormed on multiple ideas and none of them felt satisfying. I was tired of being a one trick pony with silicon and nothing really felt like it was adding up to a meaningful experience. I spent a lot of time going around in circles till I gave up and focused on everything but Physical computing.

And that’s probably the best thing I did.

Not working on P.Comp gave me time and distance to think about it and combined with the happy coincidence of my friend Nun continuously going “I wish I could use all the buttons!”, it led me to a happy place that became the start of an idea that seems promising for the final project.

So here’s my 5 minute pitch:

Update: Lillian and Atharva have decided to team up with me! This gives us an actual chance to make a nuanced, complex interactive model so we have updated our deck with the new, refined idea.

We were born in the 80s and was made in the 90s.

We were born in the 80s and was made in the 90s.

It was a great time to be alive. The music was the best.

It was a great time to be alive. The music was the best.

The cartoons were definitely the best.

The cartoons were definitely the best.

The technology was clunky.

The technology was clunky.

But with the physical controls, it was so satisfying.

But with the physical controls, it was so satisfying.

It was intimidating to approach it.

It was intimidating to approach it.

But when you got it, it became a part of your muscles.

But when you got it, it became a part of your muscles.

And the sounds really brought them to life.

And we all remember clambering to rooftop antennas to fix a TV signal.

And we all remember clambering to rooftop antennas to fix a TV signal.

But the 90s also had something awesome. Insanely obtuse point and click adventures games!

But the 90s also had something awesome. Insanely obtuse point and click adventures games!

Where the instructions were minimal and the user had to play around with the interface to discover the path ahead. (In the image above, you have to throw a bone so that the fire beaver on top jumps from the ledge and then quickly pull out your fire extinguisher on it. Once the flame goes out, you can collect a key that opens a gate. Phew!)

Where the instructions were minimal and the user had to play around with the interface to discover the path ahead. (In the image above, you have to throw a bone so that the fire beaver on top jumps from the ledge and then quickly pull out your fire extinguisher on it. Once the flame goes out, you can collect a key that opens a gate. Phew!)

So, we want to build a machine where the instructions are obtuse and the users have to play around with the interface to figure out how to make the machine work.

So, we want to build a machine where the instructions are obtuse and the users have to play around with the interface to figure out how to make the machine work.

You walk up to to a machine that has a button, knob and dial garden on its face. There’s a single bulb blinking. What will you do?

You walk up to to a machine that has a button, knob and dial garden on its face. There’s a single bulb blinking. What will you do?

Fab 2: This is a drill. Repeat.

For the 2nd fabrication assignment, we had to make 5 repeating shapes of the same dimension. I was still hung up on making lamps and I wanted to combine wood and silicon in such a way that the 2 materials are interlocked with each other. My idea was to split the wood into multiple sections and then fill silicon between them as you can see in the figure below (the black area is the silicon):


I wanted to illuminate it from the bottom and the angled cuts appealed to me more. I came up with the straight cut option as a backup plan (2 months in ITP has taught me that at least!) and set off on my merry way.

I found a piece of squarish wood from the shop spring cleaning that I cut into wooden blocks using the miter saw.

The next task was to create equal blocks which was achieved using a pencil, ruler, miter saw and the sander.


I forgot to clamp the first piece and lost the whole piece as it flew away and smashed on the wall. Never forget to clamp the wood, kids!

Thankfully, I had extra wood and the breakage proved to be a minor inconvenience.


I traced the diagonal cut shape and went at it with a band saw and sander. I got the shape I wanted but problems were immediately apparent:

1) The sander eats through tiny pieces of wood. The piece on the middle-right became smaller than the rest in no time.

2) With the diagonal cut, it becomes very hard to keep track of the perpendicular surfaces and tracking the relative position to each other.


In the interest of time, I decided to go with the straight cuts and eliminate the smaller pieces once they are cut.

Measure and draw clearly marked go-no go lines.


Put your trust and full attention in the band saw and sander.


Cut small dowels and glue them in.


Wait for a few hours and voila! You have your wood shape.


I did not get any time to pour and cure the silicon but that’s next on the item list as soon as the class is done!


1) The first prototype sucks. ALWAYS. Everything that can go wrong will go wrong. In hindsight, I should have made one and then made the others.

2) The hardness of wood varies so much that it’s not always possible to predict its actual behavior on the cutter and the sander from the sketch.


4) Sanding eats into wood fast and measurements go for a toss. The sandpaper is slower but you are in so much more control.

Bonus picture of leftovers:

(9/14) Posenet adventures.

I had quite a fortuitous moment that my project was to focus on machine learning while our final projects were due. I have always been quite wary of using anything that does machine learning as I don’t think I am don’t have enough programming chops but ml5js laid that to rest. Picking it up was so easy and getting posenet to work was trivial. It gave me the confidence to approach these algorithms and start thinking on those lines.

Back to my concept.

I have always wanted to work with the body as an input (Since I don’t move so well in real life.) Until now, most of the work related to body, gait and posture have been with a kinect or a specialized camera that is currently beyond my scope of technical ability. But posenet makes body-tracking so trivial that I realised that I could work with it.

The posture is so iconic for a person that people are defined by it.

Screenshot 2018-11-07 08.50.58.png

And who hasn’t done this!

1920px-EB_Games_Expo_2015_-_Just_Dance_2016 (1).JPG

So i want to take the energy and goofy fun of dance games, karaoke and combine it with posture and movement.

My idea: People are given famous posture to imitate one after the other and they are supposed mimic it. The computer compares their posture to the original posture and scores them on the basis of it.


To get it working, I realised that I had to work with ratios and angle instead of absolute numbers. My explorations with the posenet library was to use it to calculate angles between various body parts but I was thrown off by the vector method in p5 which refused to work for me. So, here is a sketch of my failure:

Fab 1: Welcome to the machine

For our first “Intro to Fabrication“ class assignment, we had to make a light-switch. The rules were pretty simple:

  • It should be portable.

  • It should create light.

I went through all the other weeks’ assignments and I realised that it was going to be the most open ended assignment that we have in Fabrication. So instead of working with wood, acrylic or any other material that is going to be used in the class, I wanted to use something that I will not be able to use again. I had some silicon lying around after my mid-term adventure and I was like “Why not?“. What I wanted to do was make a silicon shape that looked with no exposed electronics outside but it lights up once you touch or squeeze it.

As a first step, I cooked up a batch of silicon and added acrylic color to it to create a batch of colored silicon.

All the material in the picture above were found in the junk shop. YASS.

All the material in the picture above were found in the junk shop. YASS.

Adding the color to the silicon made it cure faster. Weird but I wasn’t complaining. Some That ITP friends pointed out a thai sweet called Khanom chan. The next step was to figure out how to cut it and that’s where I made a big, big mistake. Cutting silicon with a cold razor was a bad idea and I completely messed up the cuts.

Physically hurts to see this disaster of a cut. Ugh.

But Anyways, I soldiered on and cut a small air-pocket inside to fit the LEDs, battery and wire. the Leds were packed on top of a battery and i made a copper contact that hovers over the battery and gets connected when you squeeze the top. With some trial and error, i got it working and slathered some fresh silicon in a clamp and prayed.

4 hours later, It worked! Here’s the video:

Woohoo! Pretty satisfying except the shitty shitty cuts. Oh well, Onto the next one!

Mid-term fever.

For the midterms, Jeff did a thing with his randomizer bot (Pay no attention to the man behind the curtain! ) and I was paired with the mighty Shu-Ju!

For our first meeting, we spoke about what we liked, enjoyed and wanted for our project. We both agreed that we need to make something which involved some fabrication as we had not done any of it till that point. We also spoke of our love for TeamLab and how we enjoy data turning into Art. But for Halloween, we both agreed (It was a pretty agreeable meeting!) that we both wanted to make something which was highly interactive and involved multiple people playing together. (It’s a people festival, after all!) One of the ideas that emerged was a wand-duel game recreating the Harry Potter-Voldemort face-off in Goblet of the Fire. It would look something like this:


Two players would grasp the 2 ends and the middle section will fill up with light which gets more intense with vibration and the brightness of the LEDs. The first person to let go off the wand loses!

We decided to head our way and think of more 2,3 or multiplayer games and meet again in a couple of days. Once we came back together and started finalizing the idea, we threw across ideas to each other but nothing seemed to stick. Shu-ju mentioned how everyone was doing scary things and it would be fun to just do a happy candy dispenser. We left the idea behind and started thinking of something else to do.


During a break in our brainstorming session, We started talking about weird and funny interactive objects and Shu-ju showed me this video: We both were laughing through the whole thing and realised that we could actually incorporate softness in our Physical computing project and the idea of a candy dispenser that needed to be played with emerged.

We thought of a candy machine which dispenses candy only if you press it nicely. If you press it too hard, it gets cross, shrivels back and refuses to dispense any candy.

(It’s quite interesting to see how the final idea did not veer too much from the first sketches.)

Now that we had an initial concept in mind, we quickly realised that the sensor was the hard part. One idea was to put a force sensor inside a soft material and call it a day but we were not very happy with it. It felt like a cheap way out and we spent a day without moving ahead. Thankfully, Shu-ju was sitting next to one of the residents, Lola who had worked with silicon and using air pressure to measure intensity for her work in soft interfaces. (Wish, universe, law of attraction yada yada). We quickly booked an office hour with Lola and she gave us everything on a platter. What we needed to do, how to do it, plenty of encouragement and a loan of her sensors so that we could get started asap. Since we were still figuring out how to work with silicon, We raided the junk shop and found a discarded silicon tube. The thing was gunky, dirty and full of holes but a bad repair job with some hot silicon, we had a dirty sensor which looked like a dismembered finger (hey! it’s the flavor of the season) but it worked well for us to start coding.


Looks so gross, works so fine!

The lighting of the LED went on without a hitch and using an MAX7219 IC reduced the number of wires into the arduino to free up spaces for more wires.


This was trivial!

We were feeling pretty good about our progress as our temporary sensor and led array worked and it let us program the logic of the candy dispenser based on difference in pressure through the air pressure transducer (MPX5010 for life!) and we started our foray into making the actual air pressure sensor. We first 3-d printed a shape and then fashioned a cover to create the negative space for the air-pocket.

Shu-ju bubbling up some silicon tea! (Notice the upturned lid? See how big it is? That caused the first skin to be fragile and a total loss)

Shu-ju bubbling up some silicon tea! (Notice the upturned lid? See how big it is? That caused the first skin to be fragile and a total loss)



We printed a total of 3 times as we kept getting the wall thickness wrong. Each wrong silicon experiment set us back by 12 hours (considering the 3d-printing and curing time) and we lost a lot of time but we finally got it right and it felt so nice to squish it!

Once we had everything, Shu-ju quickly put together and enclosure and I got to work on the final coding and assembly. We had thought the structure through during the never-ending wait for our silicon sensor we really didn’t face any issue while putting it together. (Touchwood!)

The sound, mechanism for the candy dispenser and everything else fell together really well and we thought we were at the finish line all high and dry!

Or so we thought…

Because Murphy came visiting at the last moment and the nose broke with the sensor ruining the circuit (broken pins et all!) and we had to hastily put it back together at the last moment before the final presentation (Panic and duct tapes are a match made in heaven!). The thing worked but the sound wouldn’t work and it was finicky.

Here are a couple of videos which show it working (without sound):

The project went surprisingly smooth expect the last minute hiccup. It was a blast working with Shu-Ju and I would love to do it again soon. We have ordered all the materials we need for repair so I expect the poor box to live again soon. In hindsight, we should have started fabrication a bit more early to prevent the last minute rush job and relying on a functional form because we didnt have time to think it through. Overall, I was very happy with what we had and it was a great start to ITP. Onto the finals and Happy festival season!