Woman Gaga's robotic keyboard had some assist from NASA
The Grammys had an sudden performer this week. All eyes have been on Woman Gaga, as she took to the stage at Staples Middle for her tribute to David Bowie. However a minute and a half into her efficiency, a dancing piano made its debut.
As Gaga walked over to the rose-gold keyboard that was planked atop two robotic arms, the piano moved as much as meet her palms as she began to play. The setup swayed as Gaga belted out Bowie’s “Suffragette Metropolis.” The mechanical arms moved out and in to the rhythm of the track, all whereas holding the keyboard regular sufficient for the singer to regulate the keys.
To tug off that choreographed piano, workforce Gaga enlisted the assistance of Andy Robotic, a Las Vegas-based mostly roboticist and pc animator working with NASA’s Jet Propulsion Laboratory. Robotic (sure, that is his identify) repurposed some software program he had constructed to program and animate two industrial bots for the efficiency. He adopted Gaga’s imaginative and prescient for an instrument that had a brand new dimension –- it was alive. However the course of that made it attainable hadn’t been tried on this context earlier than. So when the robotic staff bumped into an issue throughout rehearsals, Robotic’s affiliation with Brian Lim, head of JPL’s Planetary Touchdown Testbed initiative, led to an answer that concerned a small piece of rubber.
I spoke to Robotic and Lim to know extra concerning the robotic star and the rescue mission.
What sort of machines did you usher in for the robotic keyboard?
Andy Robotic: We used ABB’s robots. They’re sometimes utilized in manufacturing. So you will see these robots, lots of of them, working in unison in an meeting line creating BMWs in automotive factories. We used these as a result of they’re very strong. They’re meant to function for 30 years at full velocity, at midnight with no warmth, with zero lack of accuracy. Once you examine that sort of robotic to a do-it-yourself robotic or a one-off bot, it needs to be the robotic doing mission-crucial manufacturing routine the world over. We utilized it to artwork on the Grammys as a result of it is just like a mission-important course of the place it will probably’t go down; it has to maintain shifting.
Woman Gaga performs her keyboard throughout rehearsal. [Gif from Intel’s behind-the-scenes footage.]
How did these industrial robotic arms flip right into a dancing instrument?
AR: The 2 robots moved in synchrony in order that they did not tear the keyboard aside. Think about if one robotic went too far to the left — it will tear the keyboard. We used the track itself. As soon as [Gaga] had the music executed, we timed the robots by animating them with the music and synchronizing them to maneuver in good unison. We use inverse kinematics — that is the idea of the skeletal system for any human or animal. For instance, in the event you attain and seize your wrist, your hand strikes your arm. You are not interested by how your shoulder strikes or your elbow bends, you are interested by the place your hand goes. We used that system to make the keyboard the mother or father of the 2 robots. We have been capable of animate the keyboard within the pc program and transfer it round and each the robots calculated their very own skeletal system motions to comply with the dad or mum’s actions.
So think about if I took your palms and glued them to the underside of the keyboard, I might be capable of transfer the keyboard and your arms would naturally calculate the rotation in every joint. The robots did the identical utilizing the IK system.
Industrial bots are extremely environment friendly machines. However their energy is usually perceived as harmful. Have been there dangers concerned in bringing these bots to a brand new inventive surroundings?
AR: We’ve got quite a lot of expertise when it comes to artists working with robots and interacting with them. The very first thing we do is a security evaluation. Though it appears harmful, we calculate all of the dangers; we will by no means get rid of them, however we will reduce them. A robotic by no means needs to harm an individual. It is very uncommon however any time somebody is harm by a robotic, sadly, they are not following process. As robots turn into extra prevalent, the common sense element will make them safer when it comes to human interplay. One of many causes Gaga was standing was in order that if there have been some push towards her she would not be pushed away. Defending the pop star and the adjoining personnel is the No. 1 precedence — that is with something in present enterprise.
“A robotic by no means needs to harm an individual.”
Because you have been making an attempt to adapt these robots to a brand new setting (an instrument, on this case), did you run into any particular issues main as much as the efficiency?
AR: Throughout rehearsals we had some points with the strain on the pink piano. There isn’t any precedent for this so we needed to provide you with a brand new answer. Based mostly on my work with JPL, I used to be capable of get the assistance that was wanted.
Brian Lim: At JPL, as a result of we encounter so many issues with machines, what we’re good at is recognizing what the stressors are. As a result of the robotic positioning [on Andy’s set up] was extraordinarily exact, slight misalignment for any purpose might generate extraordinarily excessive and undesirable masses. The mannequin keyboard Andy obtained for programming the robotic was considerably softer and lighter than the precise keyboard used on the stage. So, when the ultimate keyboard got here in, it had extra complicated actions that brought on the robots to battle one another, leading to load errors that didn’t seem with the softer mannequin of the keyboard.
How did you stabilize the bots?
BL: I steered that one of the simplest ways to unravel the issue was so as to add compliance. Consider two very robust machines which might be working collectively: Until they’re completely synchronized, which does not occur in actual-world physics, it’s a must to have compliance. So for Woman Gaga to be pounding her keyboard and never have an issue, the system wanted flexibility in order that the robots wouldn’t eat one another up. Within the brief time that they needed to repair the issue, I steered including “squishiness” with a skinny piece of rubber that gave some compliance between the robots and the keyboard. This allowed the robots simply sufficient give and allowed them to work collectively with out screaming, “Error!”
The rose gold robotic keyboard. [Image credit: LG] Industrial bots (or any bots, for that matter) hadn’t earlier than been used on a stage as grand because the Grammys. What have been your largest challenges in making this work for reside tv?
AR: Working in an surroundings just like the Grammys, the place we have now six minutes to primarily arrange a manufacturing unit of robots that might sometimes takes weeks to put in, is difficult. We needed to roll within the robots that have been in off place, calibrate them and convey them in control in beneath six minutes earlier than we got here again from the business and had the countdown from the director. You must institute and construct a robotic system that might be able to carry out in that time-frame in entrance of tens of millions of individuals on nationwide tv.
It is coronary heart-stopping. Plans exit the window when you’ve one band unloading and the subsequent loading in. One is acting on left; the opposite is loading in on the best. Sometimes, loading at live shows takes hours. It is very tedious and precarious. We had checklists identical to a pilot does in an airplane. And if we had issues we had catastrophe restoration plans in place.
One other problem was that stagecraft hasn’t caught up with the know-how. It goes again many years with staff who do not know what you are rolling in. They’re used to rolling in props however they are not used to bringing in robots which are very refined and finicky. Within the coming years, you are going to see stagecraft turn out to be increasingly more outfitted for robotics. Individuals who have the capability to look ahead to it and embrace it and in addition work with robotics to show them about stagecraft might be profitable. I discovered a lot from previous-faculty stagehands about how these things works.
In what methods do you assume robotics is informing artwork?
AR: It is all about amplifying the artists’ capabilities and enabling them to precise themselves on an exponential degree. The artist has to make use of robotics in a approach that’s genuine to their artwork type. In the event that they’re making an attempt to do it for the sake of robots, it does not work. However when the artist really makes use of robotics methods as an extended plank for them to step out on to precise themselves, the viewers appreciates that. It is about making the artwork higher.