Vue.js guitar karaoke: how I built it

I have created the ultimate piece of content for me personally as a devrel and I was thinking about this for a few years like how can I combine all my passions and have the right Tech at the right moment the right place to do it and I did and it worked I'm so excited I created a life and taking a few into rock and roll which is essentially a guitar karaoke run completely in the browser so I did this thing in front of a thousand people at a conference all my own no band behind me nothing else just a laptop a guitar and me plug-in go and the audience would live vote which songs I would play and everything happened really organically and it was a lot of fun and so in this video I'll show you how I build it [Music] let's first start with the signal chain so the signal chain of course starts with a guitar and that has to go into the computer so what we use for that is an audio interface that actually converts the analog to digital signal and over USB just sends it to the computer and so from there it goes into the laptop which then has a general output of audio over like HDMI or whatever you want to use that then goes to the venue's output and so this is the essential chain of where the guitar comes in but if you just do it like this it sounds really bad because it's just that dry signal from the guitar so what you need is you need to amplify it well that's what we do with this this is a tool by neural DSP and they have been working for a few years alongside a few other companies to actually make digital versions of old-school amplifiers that are also analog that sound amazing and a few years back this was possible but not affordable and now it gets much better and this was like 99 bucks it's amazing and so this also goes to the output of the computer and then of course we have the browser and so the browser itself playing the background tracks that I recorded because these songs were too long so I made them all two minutes recorded my own guitar backing tracks and then put back the original Focus that we extracted with AI so it feels like a real song but actually it's me playing most of the stuff and that also goes to the output and then of course there's this one more thing because from the refrain to the first to perhaps the solo or the outro of the song The Guitar needs to change its sound and so I'm using web midi API to talk from the browser to the amplifier to change the presets and the sounds like more echo or more volume or more you know gain and that could actually work through the browser using the midi API and so midi is a very old API that actually normally existed between Hardware devices you know from digital drum pads to keyboards that would actually talk over midi to other like Samplers and whatever or syncing things together with this midi signal and now that works in the browser and I had to just virtually create a midi instrument in my laptop and so we now virtually use midi between all these tools and it works extremely well let's talk a little bit about how this works in code so I just made a very very simple database because this is for a conference talk this is not live right this is not production code so I have each song has like a Json object essentially and so this is AC DC's Back in Black and you can see it has presets and so what this is doing essentially is um there is a preset control that you can send over midi and I want to go for preset 2 and the name should be Stadium Rock that's just the phrase that I'm using in the amp and Stadium Rock has to happen at second zero and then when the video is playing it's actually sending events right because it's just or I mean the audio so every second or I don't know how often but it keeps pushing events I'm here now I'm here now and I am made a listener that says oh I'm at 66 you know second 65. let me now send a program change a PC event from midi to go to program three and call it face bleeding solo so that's the solo right and then we go back to Stadium rock at second 106. and so this is literally just some chapters that I created in all these songs and when we actually go to the midi client here you can see it's getting the current time constantly from the audio player and so essentially just find the preset for the time and if it has set the preset it doesn't touch it but if there's a new preset that is higher than a certain value of where the time is it just sends this PC event over the midi and so there's literally it's that easy there's not that much to it and so essentially it's just watching the time find the preset and set the preset and so the only thing you have to do to make midi work is use an amazing Library called webmedi.js because the low level API in the browser is hard think about webgl it's pretty hard to use right web media just normalized it so the only thing you have to do is just okay let's just wait for web media to be enabled find my driver or find the midi interface I'm talking to this is a virtual one that I built in Mac OS everybody can do that and then sending presets is literally send program change on channel one and there's many things you can do in MIDI that I'm not going to talk about now because that's there's so much and so essentially when you look at that so this is now my AC DC here and you can see the amp here on the right actually sets on the default preset and it has like a lovely color here refresh this browser you'll see that it actually goes to you know zero seconds so it needs to actually send a midi event to the amp to say this is what you need to do so let's refresh so I just refreshed it goes to zero seconds and you can see this now changed to a brown interface called Stadium Rock 2. and so you can actually check here there's all these midi programs and so for channel one preset one it's boobie Boogie and then basically the stadium rock is preset two the solo for this song preset three so it's kind of mapped it just listens to these signals from the browser so when I go to play this I muted this for the video I'll go to like I don't know move it forward ah there we go so it just changed to the red face here so now it says here face bleeding solo and so it's just because it's that chapter in the song it changes whatever the amp is doing and if I go forward again oh now it went back to just the rhythm playing and that was essentially a really simple way for me to communicate between browser and amplifier which means I don't have to think about it it's all automated so I just hit play go in front of the stage don't look at my amp don't look at anything and just play and have fun all right on to the next I really wanted some interactivity in this talk because it would be the first talk of the day it would wake people up with loud music but I also wanted them to interact with you know what happened is happening on stage they have a bit of influence on what's Happening and so what I built is somebody goes to a life in kicking.forsale.app on their phone in the audience and they log in with GitHub and I want them to log in so I have their little Avatar image from GitHub that I can then use right and so the moment you log in you get to vote on five songs and you can only vote three times you can also just unfold or download and fold on another one and every time you vote I show you your avatar bouncing around on the screen which means if people start to be very enthusiastically you know voting and voting and voting you'll still see themselves live on the screen into huge venue of the conference which is a lot of fun so here on the left I have my slash live endpoint where I'm basically just counting the votes on the songs and you can see I did a few a few photos already in a previous session somewhere and on the right is essentially the interface that the end user will get to be able to vote so you're not logged in yet so let's just do that so I'm going to GitHub and I'm already logged in because I was already logged in in GitHub and this is using Super Bass for logging in and I cannot stress enough how easy it is to actually log in with Super Bass check this out literally I have a button that says login which is this function right I just do sign in with all provider GitHub done there's there's literally nothing else I have to do other than setting up my GitHub so people can log in with it this is ridiculous Super Bass you are awesome all right I have logged in and now see all the five songs here right and I just made it look good so yeah you have some lovely images behind when you scroll by it's just good fun and so when I hit vote let's vote on a few of them you can actually see my avatar bouncing around on the screen now and um I actually have three votes only so I can actually say you know what no more folds left let's remove this vote and add another vote and let's remove some a few more folds and then fold again so you can see my avatar you know going around and what I do is if like a thousand people at the same time do this every five seconds I'm removing the first little ball I've added here otherwise things go mental and this has to run at 60 frames per second while it's also visualizing the audio and also talking to the guitar amp right and so you could see every time I add a photo here these numbers update as well and all these photos are stored in Super Bass as well and so let's have a look at how I actually pull that off and so okay login is done so essentially we have a fold view component and all you need to know other than how it looks is song store which is Pina which works extremely well here upload the song ID or download the song ID and you can see like how many times you have folded what are the maximum Amber votes all this updated reactively with Pina and so in the songs um store here there's an upvote and a downfote and there's a few more but we don't need to go into that now it's so easy again with Super Bass I just have the user from Super Bass I have the client from Super Bass and literally I just say you know if I'm voting do I already have a thought from this user if so already voted stop it so if somebody's trying to hack my interface they will not be able to vote um and other than that I'm literally just saying okay I'm gonna vote well from votes and then insert this and I have a little TS ignore here please ignore that because this was coded really fast and I I didn't want to show you the errors um uh and that's it and then after that it just checks all the voted songs and then updates the interface with that and that's all there is to it it's a simple query if you want to download I'll just delete the row that's it so if you don't know how you have to write MySQL queries you don't need to know just do it like this and it's relatively simple and so there's one last bit which is like the moving balls right let's have a look at that one so let's go to life votes this is life votes and this is basically a huge canvas and what I've done I've actually created a real-time Channel Through the Super Bass SDK with nox.js every time a change happens on the photos table it actually sends refresh photos into my use async data function and so the refresh function is really easy and it's added by nux3 that essentially just refreshes your thing and this is mainly built for things like next page or paginated Ajax calls and so every time there's a new update to a table row I fresh do refresh and it queries again and this is how it gets this live thing going on because it just knows when a row was changed I query again and then I basically just create the photos data prop that is you know reactive and then you can do stuff with that and so then let's go down a little I have created a canvas and there are some gravity stuff going on and essentially what I'm doing is I create a ball and this bowl has a whole bunch of properties and it does request animation frame so 60 frames per second it's rendering these balls so every new vote is a ball and um it just renders it again with a little bit of math to make it kind of look like it's you're on the moon so it's not real gravity but it's almost real and then every time a new phone happens so I'm watching the photos reactive property I just you know add a new one with the the image of the user the avatar from cloudinary upload because I wanted to make them a PNG I wanted to rounded them and make them only 40 pixels wide and there's not much to it like and it just pushes a new ball that then is added to my canvas that is doing the animations all the time and then every five seconds it just removes a ball because otherwise it's too many balls it's ridiculous and so um once we go to another page we remove the real-time channel so it doesn't keep querying and there you go once you're done you go to the next song the whole thing repeats and so this is a short way to show you all the fun stuff that I've built and it's open source you can find it on my GitHub I'll put the link in the description and if you want more of this if you want me to do this on your conference feel free to reach out because this is so much fun to do um cheers and I'll speak to you soon [Music] thank you