CityJS conference talk: How to sniff out the glue code monster

hello everyone I hope you are having a lovely time at the conference in person because I'm not there in person hence the video I'm so sorry I wish I could have been there but I live in France and our country at the moment is on fire everybody is doing strikes you see riots things on fire the garbage is not being picked up trams buses trains nothing is going even planes are canceled so I really really tried my best to book my travel to London but it's like the biggest strikes of you know since World War II so it's there's just it's hard to do anyway so let's act like it's 2021 again we're gonna do a video for a conference talk so today I want to talk to you about how to sniff out the glue coat monster and yes I actually said the glue God monster so let's talk about what that is but first my name is Tim Mannix and I'm the director of developer relations at uniform and actually add uniform we are battling the glue gold monster but how that works and why we'll get into it so I work a lot in our community so I'm an ambassador for things like next GS or Golia for Super Bass but I also work at the mecca lines or I'm a member of the technical Council where we talk about what is Mac what is headless so all these things and so um my accent is a little strange that's because I was born in the Netherlands but then I live in France and I work with Americans and so if anything is strange this is why so let's get into it so the modern tech stack is actually pretty awesome right but when you start scaling it with all these headless systems and your fancy front ends it's getting really really challenging at skill and so this is a bit controversial but I don't think it's gonna work when you do really big projects with all the modern tools so let's talk about that so essentially you are the composer of your stack you're the developer or the architect or the product owner and so you just grab a collection of the tools that you love put them together and then make your Channel website iPhone app whatever but just combining these headless sources doesn't mean you're going to be successful right it's not a composable architecture out of the blue you kind of have to work at it and generally that's really hard to do so imagine you tend to start with your CMS right or maybe with your front end and you kind of connect them up this is how you start small but let's imagine now that we're going to do an e-commerce that's going to grow globally over a year well the moment you you feel the need for performance and growth you're going to add a certain certain types of products so in this case we're going to add Cloud an area because images at scale need to be presented fast you need to be able to crop a face or product or something and cloud and air is amazing for that and it's also a dam so you can store your assets there so you need to connect everything up well then you might need some search right either for searching a product catalog or whatever a Blog things like that so that also needs to be connected because all your algalia index needs to be filled by a few things well here is your Commerce let's say Shopify well your Shopify needs to be connected to your front end you need to be able to go to let's say a checkout bag things like that but it also needs to be connected to your CMS because how else do you grab information from your shop put it in a page on your CMS and then show on your front end um all the images need to go there um your index needs to be able to grab all your product information so you can search your products in a really nice way you need Analytics you actually need hosting and then all these Services would likely have to be connected to this hosting CI CD stack provider somehow to when something changes trigger a new release or do something right and so you're gonna see a change here means a rebuild or I don't know whatever you want to do there and then of course when you grow you probably need something like HubSpot or CRM like capabilities because you might want to send emails campaigns whatever that also has to be connected everywhere and you see how many things are actually now started to connect and this is relatively basic um normally bigger projects have way more than just this because they might have Legacy apis and other things that define their business right and then the real thing starts to happen you grow and you have let's say you go from Europe to the US and there they just like a different CMS because something is already built this is the realistic you know scenario so let's add in a new CMS and let's connect it to everything you see how crazy this is starting to get how you are cloning connections and like how are these things connected anyways do they have app stores do they use serverless functions that change data it's it's starting to get a little bit mad to be fair and so I dare you try to switch out one of these Services because they don't work or add something else while the thing is running this is going to be a pretty bad time and you're going to be this as a developer right you change one thing another thing Falls over however integrated this oh I forgot that the dude two years ago did this you're going to be essentially like this guy with this Dyson this fire is not going to go out and so essentially you wanted this because you have headless systems you put code together so it's open transparent but after two years of like a growing business you get to this and this you don't want and so this is gluto the glue goat monster and so essentially glucose is the thing that connects all these Services together and I mean we call this glue because it's hard to unstick right and so we are able to Define if you are using glucose somehow by asking a few questions so in your mind think about or in the chat just say yes or no if you're if you're dealing with this question so first one do you clean up that messy API response so it's data fits your front end well because every API is different right and sometimes apis change or sometimes the field is just not there so you need to add some code somewhere to map that data so your components understand it that is glucose or do you just adapt your front-end components to whatever that API gives you so you add a little bit of cleaning up stuff or lots of like defensive code in your front-end components themselves that can also work because you might not have like a back-end theme or an API developer somehow so you're just adapting locally and then there's another one so do you actually mix your domain data with design related data in your data models in let's say your product category or in your product CMS or whatever what I mean by that is this product is now featured that that's a checkbox to this domain model so it shows big on the front end well over time when you keep adding checkboxes and drop-downs to data models they are modeled with design related things and then it's really hard to actually still understand what they're doing if you remove something maybe something breaks and for that you need a lot of defensive code and a lot of cleaning up and just extra brain power and we don't want that glue sitting in between everything that code that you had to make so if you've said yes to some of these questions or like partly even partly you are probably pretty good at glucose and it's actually a fun part of coding sometimes but wouldn't you like to not have to do that and only deal with like the super fun front end or the Superfund API creation part of your job right and so let's just give you a quick example of what I think about the glucose or like what it could be and we're doing it with like the most optimistic example so essentially let's say you want a list of videos from YouTube but you want to form an API call from the playlist because you want it curated a certain way well what you do is you query and map the video IDs that you get back because there's a lot of other data in that playlist API that you don't need you just need a list with videos right so you have a bit of memory where you have an array with a few IDs well then you need to get the information about this video as well then you just go to the YouTube video API what you do is you query for each of these IDs you query the API and you get more information back but it's kind of a lot that you get back you don't need it right you only need maybe the ID that you already have you need a title and the image maybe that's enough for now and um please correct me if I'm wrong it could be that the API actually works a bit differently but this is just an example so what you'll do your few calls you clean it up again and then in the end you come up with one list of what you use but YouTube or Google's apis are really solid right everybody uses them they're cool imagine if this is a less good API imagine if it just changes or if a field suddenly doesn't exist if it's empty you're going to have to do a lot of like defensive like coding cleaning up things and if it doesn't exist you fill it up with mock data or maybe you don't that is a lot of messy code just to get this list of videos right and what if you have to go to female suddenly what do you do with this code you just toss it away it kind of feels a bit icky to have this so is a highly defensive code which fetches data and cleanses it right and so glucload creates technical debt and it tight couples specific code to specific surfaces and sometimes that's fine you need a little bit of glue maybe but too much glue and you cannot change anything anymore you cannot adapt you cannot innovate and the more Tech that you have the slower you get the more upset your developers the more developers leave and suddenly there's no longer a team boy that escalated quickly now what we need to get rid of gluedo the glue coat monster essentially and so there is a way and so the way to unglue this whole system or this whole approach with all this crazy code is something we like to call digital experience composition or we call it dxc which is yet another acronym we're throwing in your faces here I'm sorry it's yet another one but this is actually something that to me at least having worked with these large systems as a developer lead it changes the game and even if you're a junior developer or you don't really understand all this yet this is the thing that's going to help you going forward so essentially right now when you don't have the XC you're the guy on the left or maybe your boss is the guy on the left who sits on the pile of all the cables knows exactly where they go but what if this guy is not there do you know how to find these cables how to untangle one thing is pretty hard right and so what dxc does is actually give you the thing on the right which is still hard to understand but it's a lot more organized and it's more friendly to different disciplines but we'll talk about that later so essentially this is how you would build a page nowadays let's say this is your page There's a Hero component with a campaign and that's connected to a CMS well maybe the image comes from the dam system then you have another little component which is connected through algolia because potentially this is just a search result or a personalized thing that's similar to something else sharing on the page and then there's a product where the image comes from this the damn system again and then the product is selected from Shopify so you see how many connections are on one page here what do you do you have to kind of connect to them through the front end then you kind of code all these things together that's pretty messy it works but why if you change one thing what do you do your content that is going to be calling you is like hey dude can you can you just change this thing I want to have like five products here you're like yeah but I have to code something and of course there's in between Solutions but this happens more than you think and so the dxc the digital experience composition layer sits in between here so all these systems they actually connect to the DXE and then the front end connects to the DXE as well and that kind of sounds like an API aggregator but it's actually way more because essentially if you do this right you get preview of your page independently of whatever is connected to it right you get freedom to choose your stack because if there's one endpoint you can go to you know next GS next maybe we go Astro because why not right Island architecture for the win you get to choose it doesn't matter um this of course is the same thing as if you did it another way you can always choose what you want but here if you wanted to you can go to sveldkit in two weeks if you have the time to rebuild but you then actually don't have to care about how it's connected anyways because you have all the sources here you can actually add something like personalization for example without having to worry how do I do personalization in Shopify and how do I do it in my CMS and how do I do it in my search engine you can actually do that across these systems without a problem and the beauty is if you do it right which I'll show you in a sec you can actually have a no code editor on the front-end side of things that is agnostic from how you do all your code and so content editors will be very happy with you for once right and so this is the traditional page and here sits the DXE around it which will then essentially just connect to all the surfaces for you and you can talk to the DXE and it's a very light SDK which is very interesting right you suddenly have less to worry about and so this does give us a new way of how we create Pages like the separation of concerns between where data lives and where design lives where you're design driven system is actually different so we should talk about that for a sec and so we have domain data versus design data and in a DXE system this is actually separated out much more than you used to right now because such a system doesn't really exist well there's some of them like a back and for front end has part of it in API aggregator has some of it a monolith CMS as some of it but in this case we separate it out more so domain data is the data that defines your brand or your customers data your customers brand like products they sit in one place in a product inventory system or whatever you want to call it and you can use those in 10 years or you can use those next month and they keep being enriched and made better but it's very clean data Your Design data however sits on your front end and kind of you wanna maybe make something featured you do that in your front end you add a checkbox in your data model in your design data and so an example could be imagine you have a defense website right like ctjs you guys have a ton of different events in different cities well the event data sits in the CMS like venues speakers addresses times whatever this is all data modeled really neatly data modeled in your system where that's supposed to happen but now it's ctjs right so last week on the website you're going to push for ctgs London that's a feature defend well what you would do without a DXE is you go to your data model in your CMS and you add a checkbox for it to feature defend right now and you maybe add a drop down for make the font a bit Bolder and make it a red background because we can and over time the more and more this happens more of these little checkboxes and drop downs come in and so what happens if you kind of see this event in a different context if it's in a search result these checkboxes make no sense right so you're dirtying up your domain data and so when it's bigger and you have a multi-tenant operation somebody in Malaysia might say I don't need that checkbook let me just remove that because I'm cleaning up my data model because it's hard to edit and understand then somewhere in Germany your website is just dead now because the data is no longer there this causes you technical debt glue code complicated stuff so what you do your domain data stays clear and you have another system where you do all your design related stuff so things like featured event that's a title that makes no sense in CMS because it has no context it's just a little string but when you put it in your design data for that page in that context for that week it makes total sense so all the check boxes all the variants in like how does it look that's design data that sits in your DXE and your domain data stays around it so it's a very clean separation and so this benefits developers quite a bit so let's talk about that so for front-end people so your components are now completely agnostic they don't know um what it connects to your components are design system components essentially your props or your props this is how it was designed this is how it's used you don't have to worry about more than that when you have a dxa so essentially you make a front-end that is stupid I know it's kind of silly to say right but the more stupid your front and the less prone to error it is the less glue code you need to morph things so it fits your component that all happens in the DXE you can literally say in a DXE this is my component that sits in the front end and in my dxa and these props I need so you connect the props to an external system in your dxc directly to the fields and it just goes one to one to your front end your front end is stupid believe me you want that and then so when you're full stack or whoever is full stack um you can kind of if you want to connect anything to this the same way in any front end or in any framework and so you can basically if you wanted to have the DXE map all these props for you directly or if you want a little bit of glue or you need to change something because maybe you have to go deeper into whatever query it is in your CMS and follow some connections you can actually do that in a really nice way because there's an SDK from this DXE to do that and so doing things the same way is much easier with unit tests it's much less brain power everything gets better right and so if you can choose whatever front end you want you can host it anywhere you can render it any mode you want right so freedom is cool and so the agnostic stupid front end is actually pretty awesome and so this for example is my home page I recently released this using the dxc from uniform because uniform is actually DXE I can't believe I only telling you now but it is and so essentially let's just have a look at how this was built so we have the DXE which is uniform and it has a slug to a certain composition which is this page in this case the slash because it's home and so this hero component is basically represented in the front end in this is my next GS code it's a hero component with a few props that's represented in the DXE as well as a hero component with a few props and so this now has a cloudinary image but it also has like a title and stuff and we'll get there in a sec well then I have latest videos which is also represented in the dxa and for me I just connect to YouTube get the latest videos in the DXE so the front end doesn't connect to anything and then my upcoming talks um that component actually talks to contentful that's why I add these but it can also be in sessionize or any other place maybe notion where you just write down all your talks this just connects to the dxc front-end doesn't know then you have your latest writings and in my case actually this comes from def.2 because I like to write there and so when I click on one of those I just go to the external system where my my domain data lives right and so we still have these titles right latest videos upcoming talks Tim Bennett's blah blah blah these are actually stored in the DXE because they are context based to this design because when I decide not to have upcoming talks there but do something else that data shouldn't have to live in some CMS you just type it in so in that case a dxc is almost like a little design data related micro CMS but it's not a CMS it just points to stuff and so right now this is hosted on for cell with next gs13 and uniforms data I chose in this case comes from the CDN Edge that uniform has which means my Dynamic data is cached on the CDN Ash close to me so this is an SSR website and it's rendering in like 100 milliseconds or less per page which is pretty Epic and so let's see a bit about how uniform actually stores that stuff on the edge so basically what you get is there's a composition with a bunch of components in the dxc then that becomes Json it's basically just a rest API to get all that information it can either be a pointer to a thing elsewhere or it's actually all their props already just stored here and that actually then you know talks to all these systems and then whatever came out of that is enriched because this Json composition is initially is just I'm pointing to some external Services right then it goes to the CDN Edge where it actually is enriched with all the information that you need and cached there and then it goes to your front end and the front end has a relatively light SDK to just talk to that enriched CDN Edge endpoint and so there is an extra thing though because you might want to hook in to whatever comes out of uniform for example um it could be that for my cloudinary image I only get an ID but I might actually want to get the alt deck this is just an example right so you get hooks in the uniform SDK to be able to just get the initial data and then query other systems um and then enrich that and then send it to your front end so if you wanted to to add just a little glue just to do the things that you need as a developer you can and so this gives you a bunch of Freedom right and so my front-end component in this case is just a simple one there it doesn't know about anything other than these are my props this is how I render it keeps it very simple if I wanted to change this in view no problem just works and so it's not just a backend for front end but it's also not really an API aggregator because it does a few things more so I actually made like a little video of how I um edit my website you can see I'm visually editing this right so I'm adding a little list here right and I want it to be a grid so let's just um fill out a little title and so this title represents an atom for front-end component in my front end you see it directly renders and here you can add loops and I have added a data model to find my latest YouTube videos in the data model setup in the DXE then you just assign whatever comes out of that API you assign it to your props and so literally what you see on the right here are the props of my component and so you can set it to video add a title in this case description is not really needed I think but why not you can add the date and then there's adding the media and you see it literally build up over time when I add this stuff and this way you can add a bunch more things because you can actually add access like sorry like personalization related stuff changing things up so this actually is agnostic to the tech it just works with this very light SDK so developers are pretty happy with that hopefully but then content editors can visually do this this is literally my live website being edited in draft mode when I hit publish it actually rebuilts or whatever in this case nothing happens because it's SSR just has to cache on the CDN and so we went from this do this it's a lot more exciting and a lot more structured and it also helps you to like the thing is it can be as small as you want because you can essentially just have this old system and slowly start integrating with the new system and then over time you just grab a little bit more into the new system and you're you're good to go and so this is what I wanted to talk to you about and so if you're interested in this whole approach and you like what you heard uniform is actually one of the only ones to do all this there's a few other companies touching upon it like Netflix going in a certain direction but only code you have other systems that touch it but not fully and so if you're interested you can just join our Discord you can scan the QR code or just go to uniform.2 Discord so we can chat about this because it is a really exciting way for the future and so thank you very much um if you liked it feel free to follow me either on Twitter or on YouTube and thank you for sitting through a video rather than me being in person and um next time I'll be there cheers