This is a transcript of The Resonance from 2024 December 29.
Intro
0:00: streaming posting [Music] announcement oh I can hear myself oh my
0:07: God there we go posting live
0:13: [Music] streams post it there we
0:21: go what what are you doing over there come over here hello
Welcome
0:34: hello hello everyone we should be live we get messages yeah we got
0:41: messages I'm right if it's too bright let me know I'll turn it off I'm just I'm like
0:47: glow New Year's thing I guess hello hello hello J bden hello
0:55: fuzzy got people coming in uh can you hear me fine is the audio
1:00: okay this one too blah blah blah up in this thing there we go hello
1:09: everyone people slowly trickling in audio is all good thank you so I guess
1:16: we could get started so hello everyone um and welcome to the I believe the
1:22: seventh episode of the resonance uh which is um sort of like a hybrid
1:28: between my office hours where you can ask me any questions about resite whether is technical whether is you know
1:34: um philosophical like how like you know what do you want resite to do like anything with business um essentially
1:41: anything around resonite I'll try to answer the best I can uh some questions I might kind of defer to some of the
1:46: other like departments particularly for stuff like you know moderation things uh uh because moderation they host their
1:52: own office hours like you know um right before their resence actually
1:58: um and and uh you know sometimes they can look better equip can like you know
2:03: answer those questions but I'll try to answer like you know things the best I can um but also like might like to go
2:10: into more like long longer winded kind of rambl and like the high level of why things are done the way do will give you
2:17: like you know better sort of bird's eye perspective on you know how resite is
2:22: made what you want to do with it how we're kind of know approaching everything and so on um and also yes
2:28: this is the last episode of the resonance in uh 2024 this is the last
2:34: one I actually picked this world this one's from last year but caror jump I haven't published like the new one yet
2:39: but I like might be F thing it's going of PR world has like a um what's it
2:44: called a solar Punk kind of like you know um style to it and I really like it kind of VI so um H and H hello dead Dron
2:56: and Ultram uh uh so if you want to ask me any question make sure um uh to end the
3:06: message on Twitch chat with a question mark that way oh no no no it's not the
3:11: last episode of the residance it's the last of this year we'll be doing a lot more um all Char hero Godly um so um
3:21: yeah make sure like the question ends with a question mark it doesn't actually need to be in but the question mark needs to be this somewhere then it pops
3:27: on this thing like for example this one this one actually know just popped in there we go like this one you see I
3:36: can I can pull them out too um so if you do that uh oh I didn't talk correctly
3:43: let me do that again I think it's Discord being Discord
3:50: um yeah because like this Discord likes doing the thing where it doesn't show the full
3:56: name I'm just going to send another tag this
4:03: hours and I'm actually going to disable streamer mode oh I see just pick a different
4:11: thing uh just going to Ping
4:19: again correct there we
4:26: go there we go thank you me
4:32: know so then like we should be ready to start um so if you got like you know any
4:38: questions like you know make sure like to end it like with a with a question mark and I'll do the best you know what
4:44: I can answer and when there time I'll probably like you know BL like bunch of different things so with that we
4:51: actually have like some questions alreadying well there's one um so there's a bunch like uh
5:02: popping um so the first question is from F bipol bear I've been watching your
5:09: gassing spotting work with interest after trying to find ways to bring him into res night before but failing and
5:14: I'm wondering how the progress is going and how does it feel to be working with something a little more fun than necessary for a change so yeah uh in
5:22: case you haven't seen actually let me see if I can bring the video um give me a second I have it right here
5:31: so it just wrong but oh right I'm on the quest controllers so clicking also jumps
Rendering
5:38: there we put this in make sure it's in the frame so this is pretty much like
5:45: progress I made this video like earlier uh earlier today uh I've been working on
5:50: adding support for rendering gas plus and Nite it's not something like you know you could like Implement yourself
5:56: because it doesn't require more tighter implement like another pipeline um and
6:03: I've been basing my work on existing unit implementations the only problem is those uh existing implementations were
6:09: designed for newer versions of unity using some features that are not available in the one we have right now
6:15: uh so I had to kind of alternate Solutions particularly the problem was
6:21: the S the soring algorithm because when you rendering gassian splits you need to sort them based on distance to the
6:28: camera um the algorithm uh that the some like the reference implementation uses uh or like
6:35: Pro basing it on is just something called wave intrinsics and it's like a
6:40: way where you running like a compute Shader the comput Shader like different
6:46: threads can communicate with each other and exchange data uh that's not supported version of unit we have so I
6:52: had to find different sing algorithm and adapt it called the python exort and it actually ends up working fine and then I
6:58: have to like do bunch of like rling and adding our own data types for it and so on but pretty much here uh this is
7:07: rendering in our version of unity um you can see it's kind of working pretty well
7:12: um you can see um um you can see like like you know
7:19: there's like the preview there's like this one this whole thing this is already loaded through our own FR engine
7:26: class so like the loading of the gassian split uh um and then like you know pushing the data that's our own code and
7:32: the rendering is an code from the um from the open source uh project rendering them um it's pretty much in
7:39: the stage where I have it like it's hooked into cameras it needs a little bit more clean up and once that's all
7:45: done I'll properly add like new like new asset type into resite so you can just
7:50: you know Dr and drop and it'll automatically set up set up stuff on the side unit so it's actually very close
7:56: the one show stepper which was the Sorting algorithm uh that's pretty much resolved you know you can see it works
8:04: um so at this point like you know it's it's just matter like you know probably a few more days and I'll
8:09: have you might like depending well today like we like I'm kind of visiting so might be like doing some things like
8:17: we're going like to the cinema to like watch the new Sonic movie so might not happen today but like it's it's it's
8:23: very close like it's going to happen and it's going to happen very soon and yeah it's been like it's been a
8:28: lot of fun because been kind of like you know doing this kind of on the side like when we're kind of like you know hanging out and so on and we kind of just um I
8:36: kind of doing it like over the weekends as well over like past few weeks um just kind of nuding it overs and I had a
8:41: little bit more time to like P you know on something that's not really work but something that's like you know more fun
8:46: like hobby thing that I wanted to do and it's been kind of like you know refreshing because like I found myself
8:53: to be like I really want to work on this like I really want to see this and I've been like you know going around 3D
8:59: scanning everything lots of ging splits and I'm like I want to bring them in here I want to you know show them off to
9:05: people and I want to see them like in a VR in a c environment so like I've been very excited like you know to get this
9:11: thing going and get this thing finished um and I do believe like it's going to be like a cool thing for everyone as
9:17: well because um I think this is going to make us the first social VR platform to
9:22: support these so you can you know just track and drop them in and like you know show them to people um so hopefully you
9:29: if people like enjoy this as well but yeah it's been a lot of fun it's been very refreshing to just kind of work on something like where you have natural
9:35: drive to kind of do it um so it's uh um
9:42: it's um it's been it's been a little of fun yeah and I'm I'm excited to have this on
9:49: so soon Dam uh next question we have from Glen
Photon performance
9:55: VR from testing it appears that Photon does runs much slower than Unity part ad to be look at around 20 30 FPS Max is
10:02: the limitation framework of photon dust and will the update to net 9 allow it to much more smoothly faster well so it
10:10: kind of depends like I don't the problem is I don't know like you know your testing set up for this and the performance is
10:16: going to heavily like depend you know what you're doing with the syst for example with have particles that like
10:22: you know resolve collisions um they'll like run generally slower and especially with the version
10:27: of like Unity when we're still running with more um there's been some tests where we can kind of see um you know
10:36: where ESS like in a complex environment with of colliders it can run pretty slow
10:42: like you know when there's lots of collisions to resolve uh but like when when we run the same thing with net n it
10:49: runs like like 10 times faster so that alone is going to like help it a lot but
10:54: also it depends like there's like no locking like it is not like to 10 30 FPS
11:00: and it definitely like with my own testing it runs faster than that so
11:05: again like you know it's hard to talk like generally like when I don't know what your testing conditions are but I
11:11: can tell you for sure there's like no looking um if there is we need like a report like you know to kind of see like
11:16: what is happening on your particular system because sometimes these things can be like system dependent uh however
11:23: it is still like um there's still like some issues that need to be resolved um particularly with like a um um with like
11:31: a submission system because like when there's like two parts to phot on Dust one is the actual simulation and that
11:36: tends to around really fast even like with a mono the other part is like you
11:42: know pushing the data to be rendered into unit and from what I've done like
11:47: in testing that part actually ends up um that ends up you know
11:52: going kind of slowing things down and there's like still parts that like need to be kind of wrapped around uh because
11:59: right now um what can happen is like you know Photon does the simulation and then send the data to Unity and essentially
12:06: Waits until does another simulation step until you need to integrates the
12:11: result uh so like you know it's going to depend on like how fast you running how much other stuff is happening so like
12:17: there's like you know stuff can be done like put it on like you know separate higher priority queue but also improving
12:23: the performance of the submission right now we're still kind of like adapting unit this particle rendering for like
12:29: you know data we simulate um and that might be kind of like a limiting factor so like one of
12:35: the approaches I've been kind of like looking into is using actually uh computer shaders to like make our own
12:42: thing to like render M Unity the problem is you know it will take extra time to implement that um but it might be you
12:50: know needed like if like we cannot uh get the submission to the unit is one
12:55: like be fast enough uh the problem is like you know that works going to going to thrown away but also like use some it
13:02: it's like you know compute shaders also kind of funny because there's actually some overlap with how the gasan Splats
13:08: work because it's also using a compute Shader to sort of likee generate like you know the geometry to show the
13:13: particles on your screen um so like I might end up like you know doing
13:19: something like that like up think like a mechanism um so there might be like interesting Synergy so it kind of
13:26: depends like I would need like you know more kind of information and there's like some reports but like I've been um
13:32: like over the like days I really like been paying super much attention we kind of just doing fun things but I'll be like looking into it like you know like
13:39: once the New Year kind like rolls in and once I'm some once I'm done with some of the fun stuff so Circle back like the
13:46: photon dust and look into those um but yeah like the the update net n is
13:52: definitely going to make it run faster um but we also like the goal is to make
13:57: it run as fast as it can before we make the switch as well and there's things like you know that like need to be kind
14:03: of investigated still so um and like since I haven't like look into those like super deep like I cannot give you
14:09: more specific answer right now uh but yeah that's I do I do think like we can
14:15: get it like you know it's just it's more a question of like how much like
14:21: how sort of like you know like like time invested versus returns like you know
14:27: equation so it's kind of like um it's like how much time do we want to spend
Time investment
14:34: now that's going to be thrown away you know to bridge us until we make the switch um and how bad like you know it
14:42: can be and it's something you know I need to like look into how well we like sort of evaluate how much time needs to
14:48: be put into it and how about it is so like if we can say like you know we can get it to like even with the limitation
14:55: vinity like if we don't spend too much on it maybe we get it to 90% you know of like the refresh rate um maybe that you
15:04: know that's enough and we don't need to spend like you know extra time to get it over 100% until like we make like you
15:12: know the switch with the rendering engine um but maybe like you know we need to get it you know from say like
15:19: 30% to those 90% And that spends more time so I I'll have to look like it's
Performance issues
15:25: really hard to answer these questions without like having to like look at it you know individual details it might
15:30: like one of the things it might even be something stupid because what I think might be happening is just kind of ordering some of the
15:37: submissions so like one thing I have a hunch that might be happening is like when it finishes simulation it actually
15:43: Waits until next frame when it can hit the submission and it like ends up like
15:49: having the actual total submission right um but like I I don't know for sure
15:54: right now if that's what's happening or not I need to like you know run through the test and investigate that so once
16:02: I'll know more there's there actually been another like performance issue where it locked your frame rate like you
16:08: know significantly lower like it would essentially like with some existing systems when they converted to Photon
16:14: dust you would start lagging and what it like and I actually thought like it had
16:19: to do with it submission mechanism slowing them downs but what that turned out to be is that uh one of the trail
16:27: like parameters wasn't not converting properly so the system the converted system was actually generating 10 times
16:34: more geometry data for Trails like the trails were like way longer than they should have been um so it lagged as a
16:42: result of that because like it was you were like you know rendering and processing 10 times amount than the
16:48: original was so it wasn't like you know one to one comparison forhead and fixing that conversion that actually made it
16:54: not like the frame rate um so you know other kind of kind of you know stupid
17:01: things like that which end up like being relatively quick fixes and like end up like boosting The Raid like quite quite
17:09: a bit so I'll I'll what I can tell you is like I'll
17:15: essentially like you know get it as fast as I can I'll kind of evaluate how much it takes it might be you know that like
17:22: with some like simple fixes and end UPS of being like something like 90% of the performance and I'll be like okay that's
17:27: fine it's close enough we're going to keep it at that until we switch the graphic engine or maybe it'll
17:33: turn around to be like no actually I need to invest more work into the render like it's going to take more time but
17:40: it's going to bring it like you know kind of closer so but that that is still kind of remains to be
17:46: determined but yeah hope like that can answers it thoroughly um the next question is from
17:53: lexo gas in SPL thing I know you've had quite a bit of fun implementing a viewer into resonite but someone who doesn't
17:59: know much about it apart from just the way to make and VI 3D models what is it and how it can be used in the res so R
18:06: now you cannot really use them in night yet like you'll be able to soon but uh
18:12: to Simply they're they're sort of a new way of rendering I don't even want to say
18:18: models because it's not exactly a model in the traditional sense uh you can
18:23: think of like a cin plus um sort of like almost like an extension to point Cloud
18:29: where instead of um instead of like each um you know point being just a point is
18:36: like a colored blob you know it's like a three dimensional kind of blob that's kind of fuz and it can be like you know
18:42: they can be all kinds of different shapes like can actually um I go over here here give me
18:48: a we so I can like you know they all this little bit so like you know if you have
18:54: a point Cloud you have like you know just points and they'll be like you know they all kind of same size and
18:59: everything you know and like can like repr something with that but it's kind of so with cian Splats each point
19:07: there's like you know this kind of like fuzzy blob around it you know and it's going to be like 3D and it's going to be
19:14: very op here and the closer you get like you know to the the closer you get you know to the
19:20: edge the more transparent gets and it's kind of you know falling the gassian function and you have a bunch of these
19:26: blobs and like you know some of these some of the these are like you know can be like roundish some of them can be
19:31: like you know stretch I'm just going to call simplify you know some of them can be like you know like this and they can
19:37: have like different orientations and what the cassins end up doing is they end up like approximating
19:42: you know the shapes so say like you know we have like a shape like this and you know you can end up like
19:49: you know having a gasan here that is like no point and we have like another gas in here and they kind of you know
19:55: approximate that shape um both really good about them is because they um they are fuzzy and they
20:03: can have different uh you know different orientations they're really good at representing soft details like fur or
20:10: plants so you don't get like you know that kind of kind of traditional blob
20:15: you get with 3D models um the the other like benefit of them is
20:23: uh oh one more thing is they also have um directional information so like each
20:28: C uh it has a color but it color is encoded using something called spherical
20:35: harmonics and what essentially is it's a way to encode the color where it depends
20:41: on the direction so like if I look at this casting from this angle this can have one color in this direction if I go
20:48: over here and look at it from here like for example say over here is red and this is like an extreme example you
20:54: should going to be similar color but then I go over here and it's going to be
20:59: green so like it can kind of change color and it's really good for you know capturing some of the like um like
21:06: Reflections and like sub changes you know in the scene based on the direction you here from uh which helps like exra
21:13: realis but also the good thing is like you know like um uh they really work
21:19: really well with the 3D reconstruction because when you reconstruct stuff uh it uses a process called gradient descent
21:26: it's a form of machine learning where um it's essentially like learning
21:32: to like you know to approximate your input data which are like you know typically photos uh so like you know photos if you
21:39: like shape like this maybe like you know maybe it starts with a single gassan that like you know tries to cover the whole shape but it's like this is not
21:45: accurate enough so I'm going to I'm going to split this gas in and put like one here and one here and they just kind
21:51: of slowly kind of settle in place and then you kind of split into more and maybe say like maybe say like there's
21:57: like a little blue blood like here so like it adds you know another Gan here and it just ends up like you know
22:04: kind of once once you're running through enough steps of like um learning it
22:10: learns to represent uh you know the scene pretty well uh if I go back to the original cam I'm going to show you some
22:18: more videos um because of that like you know they're really really really good at like you know um just the
22:26: reconstructing 3D scenes like captured from like the world uh if I there we go I put cast in
22:35: plants so let me bring this over um so I'm going to sponsor this one like
22:43: a video here so this one for example you see uh it's one of the captures I did I I going
22:49: import more than here but this is like render using gas and SP and you see like there stuff the surface is shy it which
22:56: traditional photometry tends kind of like not is super good with and you can see the shine kind of changes you know
23:03: when the camera pans around like it is up like rooking you know pretty like
23:10: realistic you see like how the shine kind of like changes based on the direction so the call is essentially you
23:16: know to be able to bring these into the scene so you can view them you know in res you can share them with other people
23:24: um so if you have like you know ways ways to kind of like share stuff um
23:31: share stuff like you capture it's almost like little fragments of reality are a really really good example
23:36: too um you see like look at the fur because gra by accident it's kind of
23:43: hard to put with this controller I grabbing look at the fur the fur is like you know there's like very fine detail
23:48: and it's like it actually looks soft which you know you don't get with traditional you know you don't get it
23:55: with traditional 3D scanning so I just want to came out like really well like it kind of captures
24:00: like you know that kind of like softness to
24:06: it and it's because like you know each of the strand of the fur instead of like you know like being like geometry is
24:11: actually composed of the calans and because the fuzzy blobs just number representing the like really well you
24:17: should be able to see like the center of the it's also like you know little of like Supple effects like as the as you
24:24: kind of Look Around the Fur it actually has like you know a Sheen to it it's kind of shine and because of the sphere
24:29: harmonics that ends up being represented as well and that makes it look a lot more real and I got like another one that I
24:36: want to show you uh this one's are l a cool example uh so this one's like a it's
24:43: really good at St plans as well so if I play this one you see like it's it's
24:49: just like it can make three representations they just kind of look real like it it doesn't have that kind
24:56: of like Machin as that you get with like models because the mod with
25:01: models it has to like you know reduce the scene to triangles it has to find
25:07: what is the surface and the new project texture on it and with that like ends up like having an error but with this
25:13: because the the rendering primitive which are the cians the cian splits
25:19: instead of triangles it ends up being much better at they representing the
25:24: scene and one of the things like people often asking is like can you convert into models and usually tell them like
25:31: you essentially that would lose the entire point of them uh because if you end up like with a model like you know
25:36: in in machine is because the model is not as good at representing you know the
25:42: scene um I like one more example I want to show you uh this one's kind of like an extreme one this actually shows the
25:49: difference you know between casting SPL and traditional photometry models so
25:54: first this is traditional model you see it has like a surf uh like it's kind of missing parts it
26:00: actually shows where gas splits can be weak though uh so this is the scene is made from just three photos that are
26:07: like near to each other uh with photo gometry it actually construct surface I
26:12: can look at it from the side but with C PL thing the moment I look away like the
26:18: scene just kind of falls apart and you see like the whole thing is composed of like these color colorful fuzzy blobs
26:26: and because the photos only see the the photos are only from like one particular angle they're aligned so like
26:33: when you look at it from from the angle of those original photos it looks correct but when you look away it just
26:39: kind of diverges because they they can do whatever like there's no VI to kind of constraint hard supposed to look um
26:46: but it's like the they are an excellent way like you know to capture reality so
26:52: one of the things I would really want to do once we also have them is build sort of like a virtual sort of Museum
26:59: like I've uh like over like this past year I've had a chance to visit uh the Yellowstone National Park for example
27:06: one like twice I took a lot of 3D scans there like I have like few hundred gigabytes worth of like data from there
27:13: and I've been reconstructing you know the geysers and stuff and like it's been looking really good because the gassian
27:19: they're really good at like you know capturing stuff like the Reflections in the water even like the steam that's coming out of them um IED some other
27:26: museums like the Museum of Rockies but they have like dinosaur skeletons I have like 3D reconstructions of those so what
27:33: I would want to do is like you know start like kind of publishing those and it's a really good way to kind of you know share things with people it's
27:39: almost like capturing a fragment of reality and being able to preserve it
27:46: and show it to other people and that's like one of the things I really like about this like technique it kind of like you know puts you in that place it
27:53: lets you kind of feel how that place feels you know like wherever you are in the world and this I think like you know
27:58: having those in here it's going to be really powerful feature and there's also
28:04: like you know lots of research going into those like Gan SP thing is um it's one of those kind of technique that's
28:10: been kind of like on the um what's the word like when it's kind of uh under ey
28:16: essentially so there's a little more tooling appearing like you know there's like tools you can just download and use
28:22: and so being able to bring those in I think it's going to be it's going to be a lot of fun and it's going to have a lot of kind of cool application
28:29: oh P people with like scans with all stuff I mean like I already do but like
28:34: now they're going to be gas um so next question is nun will we
28:41: at some point get away to put a detail texture on a Trier it would be useful for adding variation to large Landscapes
28:47: using it I mean it's potentially possible uh I would recommend making sure that's a GitHub issue and you know
28:52: make sure like to like get v on it I don't think it would be particularly difficult out we kind in a state where
28:59: we're not touching the shaders super much because we know we want to like you
29:04: know we want to move to a different rending engine we want to move away from Unity so like we don't want and
29:10: everything we do with shaders you know we'll have to rework it's actually the thing you know with gasi in splits as
29:16: well um we have to kind of reimplement it although in that particular case it's actually kind of easier to P that um but
29:24: we had like know discussions about it like I was like I want to add this like are we are we okay like is this going to
29:30: align with the new engine but like the new engine is actually going to use compute shaders a little more heavily so
29:36: we're like yeah that's fine it's not going to be too much extra work oh oh it started a show I forgot it does that I
New Years Eve
29:43: might turn it off sorry uh how do
29:51: I sorry I didn't quite realize um I didn't quite realize this got to go it's
29:57: not the it's not quite a New Year's yet but I think it just does it every hour um is midnight
30:05: somewhere I think might actually midnight like back home but like um okay sorry I kind of lose my turn of
30:12: thir but like we were kind of like you know that's not super much extra work um uh
30:18: I um uh sorry I kind on my th but it's not
30:25: like super much extra work but like so we can like make changes and like G has been working on the consolidation for
30:30: the shaders so like it's always like possible to like little things here and there especially if they don't require
30:36: too much rework but like we're kind of cautious with it so just make sure like this get a b and um and make it like
30:43: kind of prioritized at some point the next question is from El
30:49: trendar oh real question where will R be on state where you can do everything on it from developing res itself to doing
30:55: everyday work inside so it kind of depends what you mean by that because like you kind of can develop res it from
31:03: inside itself like because it's kind of feels like a cheat way but like there's the desktop Tab and actually a lot of
31:10: the times I will you know I will be like on resite but I also like I have the
31:15: desktop open and it can be like working on some stuff so um and I even like you know run one instance from resonite
31:22: while in resonite which is kind of always kind of fun um so it's
31:28: like it depends what you mean because like if you it's very like kind of open then because you could also like think
31:34: like can you like you know update like the assemblies that are running but like at some point you have to kind of restart depending on which functionality
31:40: you attach so it depends like there's and there's
31:46: also like other parts like for example you know like earlier this year we re work the settings UI and a lot of the
31:53: settings UI like the pieces you actually see as well as the facets you know on your home Dash they were built in
31:59: resonite so there's parts of resonite that were developed in resonite itself and we want to do more of it um but like
32:07: it it it's it's just it's it's too question is a little bit too vague to answer it like you know thoroughly
32:15: because like it like working resonate itself inside of it it depends also like
32:21: doing everything you know like I don't know what you mean by everything because in some cases everything can mean like
32:26: you know you go to another tool you do still stuff there while in there it and you import it
32:32: in um one thing I would probably say is like um and it's also like what it get
32:38: is weird because like technically if you put enough effort into things you could do everything with prot it might not run
32:47: the fastest but it is a tour in complete language which means if you put the
32:53: effort into it you could technically even like you know simulate the whole CPU and you could just do everything
32:59: with it but like you know it's not going to be practical um so one thing that's going to make it a lot easier is like once we
33:05: also have like you know web assembly support uh because in being able to import web assembly modules you can
33:12: write code pretty much in most major languages compile them into V simply module bring them here and you know have
33:20: them do stuff and that can include you know being able to compile the modules
33:25: inside of reson because at that point you know we can pretty much bring any code in uh and it's going to only going
33:31: to be limited you know how it can interact with the data model and stuff uh because the web assembly is you know
33:37: naturally sandboxed um so it's um it's very kind of like I
33:44: hope like this kind of answers it but um if you you have more kind of
33:50: like kind of specific idea like when you know like or some examples what you mean like you know that kind of help give
33:55: like a little bit more specific answer uh next question is from nun yeah that
HTML support
34:00: support for HTML one I mean this also it depends what do you mean by support for HTML because that can mean like when you
34:07: say that to me that can mean a lot of different things one interpretation is
34:13: like you know support for HTML is we have a web view so you can like you know open web pages we integrate you know
34:19: some web browser and that lets you um that will let you you know kind of like you know browse web pages and you can
34:25: consider HTML support the other way to have hmal support is you know to be like
34:31: where for example there's modules in um you know Proto flag that par it and then
34:37: you can like you know do stuff like with that in proto Flags you know with the document object model or what not and
34:43: script with it or do something or the other interpretation would be like you know converting it to like native resite
34:50: visuals you know like converting some HTML and rendering with u so like
34:56: there's lots of ways to can interpret those questions like usually ask people you know be more
35:01: specific like what exactly do you mean like what kind of applications what kind of use cases are like thinking of and
35:07: that way like I can answer things more specific um next question is epic e is
35:13: asking what do you mean by our own version of unity are you saying that it's running the same version or did you make your own Unity uh what do I mean by
35:20: our own version of unity is the actually the version that there I uses um which
35:25: is like 2019.4 19f I think um so it just means they
35:31: know Dead version um because like the I think it was like in relation to the um
35:38: it was in relation um to the gas SP because the the sample project it's for much new
35:44: version of unit this so I have to like you know make sure it works with one we boost uh j4 is asking how performant is
Rendering Splats
35:51: rendering of the Splats so it's going to defin Fair bit on the Splat uh the heaviest part that I found is is the
35:58: sort thing but also one of the things I do in my reference implementation uh in my like
36:04: implementation right now is like the Sorting happens every frame you don't actually need to do it every frame like
36:10: the sample project does it every like 20 frame by default and I haven't implemented that part yet Al I will soon
36:17: but even with it's sorting every frame it tends to go like it's at usable frame
36:22: rates like uh even with an editor it was like like I think it was like running over 100 FPS along with the latest
36:29: version I've been working on but it also kind of depends which part of SP you're viewing one of the things is they tend
36:35: to be having on their overthrow and it's where it also depends on the particle SPL you're
36:41: rendering um they they tend to like um
36:46: because you might you might have like you know lots of Splats like overlapping each other so it depends you know how how big they are and how many are
36:53: overlapping for the particle sply you're looking at and it's also going to determin how heavy it is at that
37:00: partical you know situation that's like another part because right now uh for
37:05: for the rendering I'm actually not like I'm just putting the data into the GPU like into PRM raw pretty much you know
37:12: how they come from the Splat you know some kind of shuffling around but like that's pretty much how it comes like it's just bunch of
37:18: floats um one of the things I want to implement once I have the first version working is implement the compression
37:25: support which actually spe up the rendering uh and also like significantly decreases the P usage so that's also
37:33: probably going to affect things overall I would say they are heavier than like uh traditional 3 models um but you know
37:41: you have like the so like you probably don't want to have too many of them in a scene uh we can kind of think of them
37:47: you know as a very kind of like they kind of equ to having like very heavy model in the scene so we have a very heavy 3D scan you know say that's like
37:55: several million triangles you don't want to spawn too many of those because that's also going to hurt uh so the
38:01: splies are going to be kind of similar but depends on particle splies how many they have how much overlap there is it
38:07: tends to be know very variable um but the main thing is like
38:12: you know you can even PR heavy suppli you can you can get them rendering at usable frame rates
38:20: um so next question is fuzzy bipolar bear how did you import previous skins of your food Etc uh those were not g SP
38:27: the just 3D models um the were Rec constructed with like photogrametry um
38:32: they're just you know normal triangle model so um and we already kind of support importing those next question uh
Custom Rendering Engine
38:40: fun goofy question from foxer uh was the origin of the name sauce for the custom
38:45: rendering engine uh so kins he makes this thing called a scotch sauce uh
38:50: which is like no this kind of cooking sauce that's pretty like delicious um uh
38:55: as far as I know he kind of came with the recipe and it's been kind of like you know one of his kind of signature things when he kind of cooks um things uh so I think he
39:04: based it on that I think the other part is also it sounds like the source engine um so it's kind of you know Pro like
39:11: play on that uh it's also the the other part is also like I think he named it
39:17: like around the time when I was work working on Proto flag because the the
39:22: code name for Proto flag was spaget and we kind like going like with
39:28: this kind of food related thing and he was like well if you have spaghet you know you need the sauce for it you know
39:33: so like it just kind of like playful uh playful um naming but Al like important
39:40: thing it's a code name which means like it's not likely going to be named it like once it actually releases uh we end
39:47: up likeing know renaming um we renamed uh spaghett the Proto flx uh just it
39:53: kind of sounds more professional so like similar things going to happen to um uh s similar stuff's going to happen
40:01: to like you know sa as well um next question from nikun quick
40:07: question what permission do I mess with to make the suspect attackers can sit down in a seat um I'm not sure from
40:13: memory I have to kind of look it up um there might be like anchor or interaction permissions like things like
40:19: they can click on uh but um I don't remember for memory I'm
40:25: sorry uh nun also so what world are you in I'm in the 2024 Creator gem like New Year's World
40:33: um it's a it's one of the published worlds if you just search new years you should be able to find it um fuzzy bip better fuzzy blob I feel
40:42: called out don't know you'll be you can be represented by Gans I mean they're like really good at like Ines things
40:49: there go projective the I uh can you share any PLS for user inventory system rework if
Inventory Rework
40:56: any so there's a bunch if you um I should should be able to go on the
41:02: GitHub uh there's like a issue for inventory work think you might have some stuff uh just in short though well just
41:09: kind of like you know kind of things I know from the top of my head is one of the big things is going to be you know
41:15: search uh being able to know search items so you don't have to like you know navigate through a million folders uh
41:21: that's actually not even um like people often talk about this as like you know you rework but I
41:29: feel like it's a little bit like um what's the word it's a little
41:35: bit inaccurate like um misleading because like it's not just reworking the UI for some of the parts we do have to
41:42: change the backend how things work under the code because you know you you can add you know a search field for
41:49: inventory but like if the system doesn't support searching is not going to do anything uh so there's like kind of a
41:55: fair bit more work already working UI um but yeah search is going to be like one of those things um the whole
42:02: thing is going to be based on the data feeds which is the mechanism that was first introduced with the settings UI um
42:10: what it will do is that like indiv individual item in the inventory templates for it will actually build in
42:15: game and you'll be able to replace them with your own as well uh this will also mean you'll be able to have multiple you
42:21: know views of the inventory so if you want you know for example uh you know there's the let me render private UI
42:29: like you know there's there's the uh facet ancur so if you if you want to
42:34: have an inventory folder there like you know quick access on your hand you'll be able to like just put one on your hand
42:39: and you know you have for example your favorite tools or materials or whatever or like you know quick storage so you can grab something save it you know on
42:45: your hand and like then pull it out um so you'll be able to do stuff like that because it's just going to be a data
42:51: feed and you'll have like you know be able to have multiple views and filter them and do other things um so there
42:57: going to be other other things uh it's also like having functional to like being able to like move and copy items and rename uh so
43:05: it's going to be both um into that we I would also like to do like you know a bunch of things like when you click on
43:10: specific items like for example photos you can preview them you know some like videos and so on so you get like a thing
43:15: that pops out and says like you know we have a bunch of actions you can do with that U maybe have like you know lot
43:21: panel that kind of pops and says you know this an item has like you know this many textures you know there's like some
43:26: metadata that has this many triangles and so on um um so it's like um you know like
43:36: there's like a lot of kind of like features as well and just kind of like reworking in a general like use uh use
43:42: the new kind of modern systems for building like UI so both our team can actually make the UI in game instead of
43:48: you know it kind of being hardcoded which is very painful it doesn't lead to like super good UI and also so you can
43:55: like you know mess with it and like build the function part of the work is also just modularizing the whole work so you can
44:01: like you know you can build um um you can
44:07: build um you know your own UI like in a mod Manner and the content team is going
44:13: to or you know they're going to make um they're essentially going to make like you know the actual visuals for the
44:19: UI while on the engineering side we focus more on functionally so pretty much going to use the same approach we
44:24: use you know for the settings UI to do the inventory as well um this is like
44:30: you know lot of like niceties because you can also then feed lots of data for example when you save an item you know
44:35: the actual item can show you know this item is currently sinking and it shows this progress in the inventory because it just it all feed the template you
44:42: know that data up thinking so there's going to be lot of like new cool features I checking out like the
44:49: inventory UI like on GitHub like there should be information I don't know like how I posted one there but um there
44:56: should be some but hopefully this kind of gives you you know some idea like what's like planned uh what's planned
45:01: for it next question is uh AES W was asking
Fractals
45:07: uh it's a fractal basically getting smaller smaller squads well I wouldn't call it a fractal because fractals like
45:14: um they essentially how to put it like they keep going no matter how much you zoom in if
45:21: it Gan Splats it'll stop you know like there's there's only so many Gans you can
45:27: have so like it doesn't really represent a fractal like um it's just a different
45:33: kind of primitive um you could like you know make something that renders fractal
45:39: using Ain plus that would be to be extra thing you know the same way like think about it like you know if you have a
45:45: picture you can have a picture of a fractal but if it's just a picture and you zoom in you're just going to see the
45:51: pixels like there's not going to be any more details because like you know that fractal got kind of baked in into this kind of primitive which are
45:57: pixels the same thing you know cians if you represent the through them if you zoom in you're just going to see fuzzy
46:04: blobs there's not going to be any more detail unless we have another process but once you do it generates like you
46:11: know more gassian kind of like more gassian to kind of show you the detail
46:17: that would have been there but gassian themselves they're they're not
Gan Plus
46:23: fractals um next question hes is asking here's a good question could you use Gan
46:29: plus to get scale uh tracing for regular model making from Source material sounds like more realistic way converting
46:35: certain spots to a game engine or 3D printable asset so they're actually not as good for um making models unless use
46:44: them as a reference like if you use them as a reference and you manually model things then yes but the SP themselves
46:53: like if you um if you remember I can actually just SP it again uh if you remember like this
46:59: showcase um so this is like a really good sample of it the Gans they don't actually need
47:05: to follow the geometric model they just need to look right you know for specific
47:12: angles so this this is normal like model and you can see if I look from the side you actually oh my God I'm not used to
47:19: the quest controllers um if I want to point you see if I look from the side you still have like the surface it's
47:26: kind of because there's so so few photos but like it it has you know coherence you know of the model so that
47:34: that is that those traditional models they're kind of useful for that it cting SP it looks fine but the moment you look
47:40: at it from left side you see there's no there's no coherence it doesn't look anything like the object is
47:48: M um so like you couldn't like really get a morel out of this because like
47:54: Gans by the nature they don't necessarily represent the model
48:00: like they often can like they like you know the C can be in the place for the model but they not guaranteed it's not
48:05: part you know of how they work um so if you wanted to like do like 3D printing and so on like if you want
48:12: to 3D print your scan doing traditional photo geometry that's probably much better B than using aan SP
Shaders
48:21: thing so hope that can answers that question um next next let's
48:28: see happy new hour I think that was when was the New Year's thing
48:35: uh Q is asking will issues with the current shaders be fixed like being not able to override values for metallic
48:41: smoothness after setting metallic map um I make good Hab isues like anything things can be fixed uh we just need to
48:48: like know about them and we need to make you know sometimes you need to like no enough people are interested in this for
48:53: us to invest time into it because you know there's lots of issues and we can only work on
48:59: some so many of them so like some will be fixed some won't be fixed um
49:04: depends um make make you know make sure make their like it have issues
After performance
49:11: um so next question from Grand UK is after performance what would you want to
49:16: work on FR so question want I mean there's multiple levels of one because
49:22: like um I always like want to work on lots of things for different
49:29: purposes and there's things I want I want to work on because uh they'll be
49:35: good for resonite and there's things I want to work on because they're also in a way good for resonite but also like
49:40: you know um they're more fun so it kind of depends you know which
49:48: angle you look at it from um if you look at it from the angle of like you know I
49:53: want to work on things to improve resonite everyone I'm like this is what resite needs and I want it to be doing
50:00: good and I want more people you know to come in and use resite so you know we
50:05: can kind of grow uh I would say like UI is definitely high on the
50:11: list I kind of want to re workk Mari because like I um I'm myself kind of
50:17: like you know like ran into the issues with it because I use you know my users Anno a lot and all the issues like
50:23: you're having withi I'm having withi and I do want to improve those because just going to like you know make is make the
50:29: quality of life easier and also it's going to make it more friendly to users because that's one of the complaints you
50:35: know we get um do I guess confusing so that's one of the things I would really want us to spend time on
50:42: um there's the ik is probably a big one too because I do like use full body or all
50:50: um and you know often times have the issue like you know with the head moving and like it's kind of scrunching up weird so like I wanted to feel good when
50:58: I use it myself but it's also like one of those things like there's more of
51:03: a work thing so it's more like I kind of want to work on it but also like there's that work pressure on it so and it kind
51:12: of makes it a bit different like for me
Workshop
51:17: um so it just it depends like you know for thing I would say like it is that these ones there's a like thing that I
51:24: feel like you know we need and I kind of want to work on them and I want to get them in as early as possible because I
51:31: feel they're going to have a big impact one of them is also related to the UI and it's having the workshop being able
51:39: to um you know essentially share any item like you know avatars tools you
51:46: know like whatever gadgets like you make uh you know materials sound effects you know
51:52: textures like whatever like asset or even like entire complex object so
51:57: um because we have like Community like you know that makes a lot of cool stuff all the time but right now the sharing
52:04: is kind of limited um it's very like you know like you have
52:09: to know a person who gives you a public folder and like you know maybe this public Republic folders and it's kind of hard to discover things so having a
52:17: workshop I feel like would be a really big Boon and I wouldn't want people to start like submitting things to it early
52:23: to kind of build a big library of publicly Shir stuff that's like searchable that's
52:29: filterable um and I think that will also help you know lots of new
52:35: users um there's like um like for example you
Future plans
52:44: know like often times when somebody comes in they want to bring in an avatar and often times you know they have to
52:50: like import one but like if we have Workshop we can just be here's all the public available avatars you know we
52:56: just filter the category we make it easy they just click on it and be like okay you have it equipped now so I think you
53:03: know having that will also make it make the kind of like initial flow for new users much better because it's going to
53:10: make it easier for them to discover you know cool content discover avatars find the stuff they want so that's something
53:16: I want to work on as well um you know for that purpose those
53:22: like things I want to work on to like you know improve General kind of like you know usability
53:29: of resonite like one of them is going to be you know prot collection support because that's going to be huge that's
53:35: going to like you know explode what you can do with it um there's you know one
53:41: of the things I really look for is having like you know the DSP in Pro like Proto so you can do audio processing and
53:48: mesh processing texture processing because I really like those kinds of systems and some of the ground workor is
53:53: already laid out it's also having you know the nested not and so on so there would be like you know really good for
53:58: people who develop stuff there's also like you know creative tools like one of the things I would really look forward
54:04: to working on is like you know the tering system um so the tering system that's
54:09: going to be another big thing that like I think will be like a lot of fun to work on uh and
54:17: it's actually similar in the particle system because I really love particle systems I'm kind of looking forward to working on uh Photon
54:23: dust but it kind of brings it back you know the thing like where it's like there's this work pressure on
54:31: it which kind of makes it a bit weirder um I don't know this like the
54:36: other thing like I really want to see is like you know timeline support actually I might I might uh do this one on camera
54:43: like to kind of show you some of the things that will be possible with this um so I'm going to move
Timeline
54:51: here I'm kind of using this kind as as an opportunity tramble I need to clear
54:58: these uh I not I'm not using the board so like I have to clean the
55:07: manually so one
55:14: more so one of the things I really do look forward like to working on uh is
55:22: the timeline system and what it will be will essentially do is offer a
55:27: generalized system to sequence anything like pretty much anything in resona it's
55:33: going to be built around you know the same core Primitives that you are they can access with inspector they can
55:40: access with uh Proto flx and I think it's going to make it a really powerful mechanism that just going to integrate
55:46: really really well with lots of different systems and open lots of new possibilities and the way it's going to
55:52: work like in general um let me make sure I'm like on the camera is you'll be able to like you know create a timeline let
55:59: me actually unplug my headset because I canot getting cut in the cable so I'm just going to draw a
56:05: timeline so see like you create a timeline and it's like you going to be like this object it's going to have a start you know it's going to be some
56:12: marks um and what you can think of as timeline
56:17: is just is it just a general you know it's concept you have like you know you
56:24: have points in time and you can put things to happen or think you know with a point in time and then you can play it
56:30: back so for example so you have like you know you have your play cursor here and you can play it you know and like as it
56:36: plays it kind of goes through the timeline and you can you know see and move around you know and that gives you
56:42: information where you essentially have like you know this is the current time
56:47: and this is you know the current speed of playback um and that can be fed into
56:53: lots of other system the other systems was going to say I'm I'm at this point in the timeline you know for example um
57:01: you know like you're going to have a thing and it's going to be here and this is going to be act whatever this is it's
57:08: going to be active you know when the time cursor is here and then you're going to have like
57:14: you know a position within this you know that's depending on where the
57:20: cursor is relative to this item wherever it is on timeline and the real power will be that like you know this
57:26: can be anything to give you an example say you want to animate you know some colors in the world this can be like a
57:34: gradient so you put a gradient on the timeline and then you have like you know
57:40: so you have like an item in the world and you have like you know like you have a drive for its color I'm ding it
57:46: wrong and say like you know you have another note so you can or module like
57:52: you know there going be probably multiple mechanisms but let's think of it as as a node so we have a gradient
57:57: this is you know some kind of color I'm going to you know just do whatever like imagine there's more
58:04: colors in this than yellow uh you plop this on the timeline and then like you have like thing which actually
58:12: feeds this and it gives you whatever color is at the current playback and then you're going to drive
58:18: a color you know of something and then you can for example say I want to drive a size of something or you know so you
58:24: can have another thing and and says the size is one here and it's going to be two over here and then you're going to
58:30: you know you're going to feed this to another thing so uh you know you get the size you feed
58:36: it like you know you drive it and as the timer kind of goes you know it's going
58:41: to it's going to change you know whatever values you get for here and you can map that to size and either you know
58:48: there's going to be component which Maps it directly so like you know you don't need to like you know drive it specifically just say like drive this
58:54: part uh um or you know you can sample with prot you can do extra stuff oh I
59:00: didn't expect that to happen but you know you can you can sample it and do whatever extra stuff you with like know
59:06: whatever processing whatever mechanisms you want to F so that's like one of those things
59:11: you'll be able to do uh kind of works can undo things like this here we go Um
59:19: this can also be you know for example audio so you can put like you know an audio here I and maybe like you know you
59:27: can overlap multiple of them and then you're going to have something where you have like you know on audio like output
59:32: I'm just going to repres L like speaker uh and you have got you have a module
59:40: which you know feeds the it it Blends these audio sources together and you
59:47: know and feeds it you know into your audio output so it's going to you know play and blend whatever audio you have
59:54: there so like if you want You Know audio to happen in synchronization with some volue changes you'll be able to do that
1:00:00: you can place it on the same timeline so you can have like Know audio
1:00:05: clipse one of the things you'll also be able to is you know um if you have this
1:00:13: is you know it can be like just an output in the world maybe you know it's like you're making like a virtual experience you don't want sounds to
1:00:18: happen a particle times but also you could just use this for audio composi thing so uh we could have tools where uh
1:00:28: if you have a timeline with audio clips on it you're going to say I want to render this out into single audio clip
1:00:34: and you essentially bake it and it's going to render out a new audio clip that's you know literally whatever this
1:00:40: is so you could essentially use resonite for audio editing because with those
1:00:45: clips you know you'll be also able to like you know um the more powerful the modules that go on the timeline the more
1:00:51: you'll be able to do with them so you could for example say this can be different sizes you know I want to
1:00:57: stretch this audio effect or maybe I want to like you know cut it so it's only part of this audio effect so you
1:01:02: could you could use the timeline not to drive something in the world but use it as a workflow tool to you know do audio
1:01:09: editing uh and you know bake audio and you can then export this you know use it even outside of resonite or use it
1:01:16: within resonite the main goal is like you know make the tool as powerful as it can be as versatile as it can be the
1:01:24: other things you know with the timeline system um you know you it's most the
1:01:31: idea is like whatever whatever can be like time based you can place it on the timeline and you can use it to drive
1:01:37: whatever things you want you know the another thing you could place on the timeline is um actually let me back up a
1:01:45: little bit I'm going to mention another thing audio thing because you might have you know multiple
1:01:51: audios but also you're going to say you know this track is you know this track
1:01:57: goes to this and maybe you have another speaker here in the world and you're
1:02:03: going to say this track goes to this so when this whole timeline plays when it
1:02:10: gets to this point it's going to play sound out of this one and once gets to this point it's going to play this sound out of this one so you can use it you
1:02:17: know to synchronize multiple sources as well um again you might have you know
1:02:23: not just aor CLI but also video maybe you want you know synchronize video there and once we have like you know
1:02:29: better video decoding maybe you'll be a still like you know composite video like you know so you can use it for video editing it's
1:02:37: probably going to be further down the line because it requires better like video kind of support but it's you know
1:02:43: by making the timeline a very generic construct it'll open those possibilities you know the more functionality comes
1:02:51: into resonite itself and F engine itself the more it it will kind of resonate
1:02:57: with the timeline and do more things you'll be able to do with timeline um these things on the timeline
1:03:04: you know it can also be animation so for example say you have like you know you have a Hol rig so you have like a r here you
1:03:13: know this thing and then I have like you know more cap animation you know and this this is already kind of like
1:03:20: animation within itself and you're going to say at this point this is going to drive this thing
1:03:26: so you know you can place you can place animations there like you know there can of B animations and what I think is
1:03:32: going to be even more powerful is you'll be able to like you know take a whole timeline which has a bunch of stuff and
1:03:38: say this whole timeline is its own thing that's kind of
1:03:44: like you know big that has like you know some outputs so like I have like a timeline
1:03:50: you know this has a bunch of stuff I'm going to put this timeline into another timeline and then you know maybe I'll
1:03:56: repeat it you know and do other things with it or maybe I put like just part of it and then like use it to you know
1:04:01: Drive other stuff so you'll be able to like new Nest timelines uh and do like lots of stuff
1:04:07: that way the other thing that's also going to be powerful with timelines uh are sort of events so you could for
1:04:14: example say um I'm going to put an event at this point and say you have like a
1:04:20: you have like a perlex and this event is going to send you you know an Impulse so like once the
1:04:28: cursor reaches here it's going to send an impulse and it's going to trigger whatever you want so you can you know for example I don't know trigger par
1:04:35: Cliff happen you know whatever mechanism you want
1:04:40: um the whole timeline you know it's not going to be tied into any
1:04:47: specific you know system it's going to be a general construct and the goal is
1:04:52: to make it like you know so it kind of integrates naturally with lots of systems in there on I anything where you
1:04:59: want something to change on time once we the other example is you know once we for example have the audio
1:05:05: DSP um you could you know have you could have like a system say for example you
1:05:12: know a node that's like you know accepting a bunch of inputs and it's like you know it's doing some kind of
1:05:17: signal thing um you could have a timeline where we have like you know sequence of you know values and maybe
1:05:24: like within you know if you open this up you know this could be like a curve or
1:05:29: something you know maybe it's like and it's like you know bazer you know maybe there's one here
1:05:37: like you know so this is here and then like you have like bunch of more modules here that you place and you know you do
1:05:43: stuff and then like there's another one here and another one here and you're got to say this PLS here this PL here this
1:05:51: PL is here and as the audio DP is like you know gener audio is actually
1:05:57: sampling the timeline for the inputs so you can use it as a sequencer for making
1:06:03: music or making sound effects the goal is to have the system be super versatile you know be
1:06:11: able to plug pretty much into anything the way you know Proto flux plugs into
1:06:16: anything the way inspectors plug into anything the timeline is going to be you know very similar anytime you have a
1:06:22: value that you want you know change over time time you can sequence it on the timeline and then you know wire it into
1:06:29: whatever you need to do um and create like you know lots of kind of complex Behavior so I think once we have this
1:06:35: mechanism it'll open up like lots of really cool things like you could for example make an entire world uh you know
1:06:43: this like music synchronization and what do you do let me just undo all of
1:06:49: this could really use the clear button oh I don't have enough f stps
1:06:55: don't know I have to do this manually um let me get rid of this of this uh so
1:07:03: say like you know you just plop an audio track you know and you have you have
1:07:09: audio here and then like you know you sequence whatever you want to happen you know whatever stuff you want to drive we
1:07:15: just you know sequence it with it using the timeline and plop whatever things and now it's going to be perfectly
1:07:21: synchronized you know with your music or maybe instead of Music this is a video and you use this you know to make meta
1:07:27: memes you just say you want we want this thing to activate or want this thing to change just sequence it it's going to
1:07:34: make stuff like that like way easier to do um like I mentioned earlier it's going to open up new possibilities like
1:07:41: you can use as a workflow tool you know do audio editing you know clip the audio clips arrange them blend them and then
1:07:47: like you know um render it into new audio clip stuff like that um the goal
1:07:53: is like you know like I said everything in resonite is make it as generic as possible so like it both integrates with
1:08:00: systems and lots of other systems can like you know build upon it uh so that's one of the things I
1:08:07: really kind of want to work on at some point or too because I think it's it's going to explode you know lots of things
1:08:13: you want to do with there night um there's another
1:08:19: um I was going to mention um I was going to mention something that
1:08:26: going to slip my mind now um but yeah this is this is I think this
1:08:32: is going to be really powerful oh remember one of the things I would like to see like once this is in that I would
1:08:38: like to come in is like you know when people actually make uh the used to make like you know this kind of like stop motion things like
1:08:45: say we can build tooling to also help you know work with a timeline so you
1:08:50: don't have to just you know work with a timeline manually sequencing things but say say you have like you know an avatar
1:08:55: in the world you know so like a stick figure and that there used to be like stick figure animations so you can have
1:09:00: tooling which says put the current state you know like I'm here on the timeline right now put the state here and it's
1:09:08: going to you know it's going to make a key frame and they're going to move it here and you know you're going to like
1:09:14: move this you know I didn't make it enough but like you know imagine his hand moved put the key frame here you
1:09:20: know and you just kind of like do animation step by step and then you can you know play it back so we can make
1:09:26: tools that let you know do stuff in the world and then capture the state into the timeline so literally make the
1:09:33: changes want to happen and then make key frames and one thing I would love to see is like you know people making sort of
1:09:39: like these stop motion animations like you know like the stick figer ones but now in VR you know with avatars and
1:09:45: everything so like and this one of the reasons I really want to work on this because I think like you know that can be a lot of
1:09:52: fun for lot of people and can like make res into not just like you know a social
1:09:57: platform but something people use um something that people use to make uh
1:10:04: cool content uh that you know exists outside of ARA so that's i' I've kind of
1:10:11: use your you know question for another thing I really wanted to talk about because this is one of the big things I
1:10:16: really look forward to like implementing at some point because I think it's going to be big for res night um there's a
1:10:22: bunch of others too so like I'm I should probably get like to other
1:10:27: questions but there's a lot of stuff I canot work on and there's stuff I want to work on you know for different
1:10:34: reasons um there also like stuff that I just want to work for like for personal things
1:10:40: um I I don't think I can actually remember like one thing I do want to add is like face tracking for desktop
1:10:46: because I want to like you know have some face tracking and I'm likeing Des all as well but it's one of the smaller things actually do I do oh there's a
1:10:54: bunch of other I just like now to think about it but if you go on the GitHub um
1:11:00: there's a project board called FRS Feelgood issues and there's a bunch of them like one of them one of them that's
1:11:06: a smaller one but also that I think is going to be really powerful are spal
1:11:11: variables and it's actually something like where you can Define variables to change in space um so you can for
1:11:19: example say like there's an area and the variables this value here and this value here and you can sample it at any point
1:11:25: in space and we have like an know helper module so like you just drive a value based on where the object is in 3D space
1:11:32: uh and I think that's going to be really powerful mechanism that's going to enable you to do lots of cool things and I really want to work on that one um
1:11:39: also physics going to be a little fun like there's a lot of things it's kind
1:11:44: [Music] of um it's it's it's there there's a lot of things I do recommend checking out it
1:11:50: board I need to kind update it because I didn't really get to work on many things on that but
1:11:56: um the gasan SP are kind of like one of those things that's kind of like you know more fun thing there like few
1:12:02: people but like the main reason like I'm kind of it's just sort of like mental health bre project so there is no
1:12:09: projects I want to work on like that there's also like projects in there I I really do want to get into some of the
1:12:15: making some of the music morals to publish some of my scans and gassian spot thing is kind of aligned with that
1:12:21: so like I want to put work into those two so hope I hope like I don't remember
1:12:27: foring for this one but it's also kind of part of the stream so I hope that gave you some idea on some of the things
1:12:34: that are going to be coming in the future uh and also is asking wa support in resite yes that's actually that's the
1:12:41: main way now I've been kind of watching web assembly for a while and it's pretty much reached to a point where I'm like
1:12:46: yeah this is like for a while I was like thinking for like you know traditional scripting in resonite maybe we'll
1:12:52: integrate Lua but I don't really like Lua and it's very like you know I was thinking python but Python's
1:12:58: harder to integrate maybe like C but it's also harder to like sandbox and it's like web assembly is pretty much reached
1:13:05: the point where it is usable outside of web 2 and it's meant to be used outside of the web to uh is designed to be
1:13:13: secure but like what's really powerful is you can compile existing code into it
1:13:18: like you can take a library written in C or C++ and just you know compile the
1:13:25: assembly and then like you know glue it with other sub and like you can now use all the existing code you could take you
1:13:30: know emulators that are like already coded and compile them into Vib assembly and then like you know just integrate
1:13:36: them with whatever you want them to um so that that's going to open up a lot of
1:13:41: like really Co possibilities as well and this is actually one of the things I do want to also work on too but it's not
1:13:47: big one too uh but yes we um web assembly is how I want I to support lots
1:13:54: of other language is because you can use you know C and C++ you can use um you
1:14:02: know like say like you want to run python or Lua you can compile you know Lua or python like you know runtime into
1:14:09: web assembly and then use that to run it uh there's lots of other languages that compile to it as well you know round you
1:14:15: know there's also C like ones and like um there's lot of possibilities there and the tooling is you know keeps
1:14:22: growing next question is CLA in with perlex is
1:14:28: there going to be an update where you hover or select the perlex note and there's information about it I mean um
1:14:33: we don't have like super specific plans but we do want to integrate documentation with resite so it's can
1:14:39: accessible and that would be like you a good way like where it can integrate into the component or like node browser
1:14:45: so you can like easily access information so maybe eventually and not super specific plan but generally
1:14:53: yes uh next question gruk is there any plans to support authorization and headers with HP get post and in a secure
1:15:00: way for Secrets um so this one's a little bit tricky because like we cannot put the secrets
1:15:07: in the world and needs to be done in a way where you cannot like you know abuse it like where other people cannot abuse
1:15:14: it because you don't want your secrets to leak and you don't want other users to be able to um you know abuse it to
1:15:21: like get access to your secure stuff so one of the things that like might be
1:15:27: done like for that is like the secrets are going to kept in user space and actually it's going to you know it's
1:15:34: going to like you're going to have like you know some kind of UI to maybe manage them or something and whatever you spawn
1:15:42: will track if it's like you know spawned by you and maybe like you
1:15:47: know add mechanism so you can only have like the secure Parts like locally so other people cannot mess with it so
1:15:53: there like general idea like how to handle stuff like that but it requires
1:15:58: extra care because of the potential for abuse and security HS
1:16:04: um next question is uh can are also thank you for the donation for the bit
1:16:10: uh got any new worlds to Showcase I don't actually have like well this one's kind of new year but this one's from last New Year I don't uh I haven't like
1:16:19: look at many worlds like recently I've been kind of like busy like stuff so I don't have any to showcase unfortun
1:16:26: gently uh next question from uh time um shenzo are plans to provide
1:16:35: support for people to Bright their own fragment over shaders stand line or enable inability with sign distance
1:16:40: fields and Z things like G spotting and break away from traditional triangle measures so you will have uh ability to
1:16:48: make your own shaders uh probably initially with Proto flx um once we switch graphic extension
1:16:55: because of the unit is very limited um like we cannot like really compile and
1:17:01: upload new shaders at R time uh at least like not in any insane
1:17:06: way um you might not be able to like write specifically fragment of vertex
1:17:12: shaders because um those are very specific and one of the things that's really important to us in the resonate
1:17:20: is long-term compatibility and there's uh the shaders
1:17:26: are probably going to be know obstructed enough to make sure we can ensure that longterm
1:17:31: compatibility um the reason for it is actually like the uh the new engine
1:17:37: right now it's planed to use mesh lights which means there's actually no vertex
1:17:43: Shader um what it means is um if we let
1:17:49: you write specifically ver shaders either one any shaders you would
1:17:54: right would break once we switch to engine which is something we don't want to happen um or we would not be able to
1:18:03: switch to a new engine which means would be effectively stuck you know with development um which we also don't want
1:18:10: so if you provide a you know more abstracted way for example to modif modify ver data that's something we can
1:18:16: W around and make it work you know with whatever pipeline is in place um so
1:18:24: probably it's going to you know there's going to be an obstruction layer very likely uh but you'll be able to like you
1:18:29: know do that kind of stuff eventually like um so hope that kind ANW the question
1:18:35: this going to be like part of the general you know support for making an or shers uh next question have you thought
1:18:41: from Navy 3001 have you thought about buying license for standable uh or using some
1:18:48: of it for resonite um I mean people use it like uh I don't really use it myself because I just do body um I don't know
1:18:56: what you mean by using some of it for resonite like like that be like very different kind of Licensing like if you
1:19:01: me like integrating the code but I don't see the need for that like we probably need a license it you know with the
1:19:07: crator and it would be way more expensive than a standard license um assuming they're even willing
1:19:13: to like you know do something like that but even like I don't know the reasoning because like you know people can just
1:19:20: buy it and just use it like it doesn't doesn't need to be like integrated like directly with this
1:19:27: I um SNB 8272 question about when will
1:19:32: be getting easier access to linear curves on Fading colors alpha or color over lifetime so if you mean like curves
1:19:39: that requires the in the inspector UI to be reworked uh because we need to actually make UI for working with curves
1:19:46: uh so probably sometime around then um the inspectors they're going to
1:19:51: be rewarded also with the data feeds so it all make it easier to make more
1:19:56: module specifically you know to work in with curves and so on uh the curves themselves are probably going to be data
1:20:01: feeds too um so sometime around then like uh must
1:20:08: thing happens uh similar with the colors you know like it needs like a gradient like editor which is own its own like
1:20:14: Unique Piece of UI uh one thing that you can do uh with Photon dust is you can
1:20:20: actually use textures so if you already bake your transition into a texture um
1:20:27: you can you know plug that in and that's going to work uh next question is going okay web
1:20:34: assembly support is added what is mean there would be a need for library or create for us programs chat in various
1:20:40: language to interact with resing specific elements would there be a reference library in one particular language for others to refer when making
1:20:47: in different language yes um we pretty much like need to provide like you know for the common languages we'll need to
1:20:53: provide um sort of like pretty much a reference library which has like in a bunch of API
1:21:00: functions and so on so you can just include that and make sure you know whatever code you're right um can you
1:21:06: know interop with um you know right uh
1:21:11: going to have like you a bunch of prop meths too and like other things um we might like do ones you know for the
1:21:18: common languages you know maybe like C um C++ or asked um but leave the rest of
1:21:25: them you know for a community so like this probably going to be something that like you know we even like you know make
1:21:30: those like opens people can kind of look at them and use them as a reference and if you want support for more languages
1:21:36: you know we'll probably like um the community help so for example if somebody makes a python integration
1:21:42: they'll have to like you know make the python side as well um and you know build it on the c or C++
1:21:49: one and gr is asking how much do you think collections will cause a wave ofing Creations a lot I I think I think
1:21:57: it's like one of those things like where people don't even realize you know some of this stuff like um let me actually
1:22:02: give an example like uh you know the brush I'm using uh if I can I don't think I can grab it from over there uh
1:22:10: me let me go for the brush where's the brush come
1:22:16: on um so one thing you might not even realize um I'm going just clear this up
1:22:23: the way Rush is work it's a really like I'm I'm kind of proud of
1:22:29: like you know how the whole system is designed
1:22:35: um let me delete these so whenever you use a brush what a brush actually does
1:22:41: under the hood uh when you start drawing it it adds a new procedural
1:22:48: mesh so like this is literally just a procedual mesh and what the brush is doing it just it's spawned in the world
1:22:54: but then as long as you keep pressing trigger the peral mesh it has an array
1:23:00: of points potentially colors and other things and what brush is doing is just adding new points to it and the Mage is
1:23:08: like you know oh I've been modified I'm going to update my own geometry so like the brush it has no like technically I
1:23:15: doesn't really have much idea about the brush stroke all it's doing is just adding new coordinates you know based on
1:23:21: where the brush currently is so it's Tech Al two systems that are mostly
1:23:27: independent from each other it's the procedural mesh system particular this type of procedural mesh and under system
1:23:33: that's like you know just adding data to it and as you know how is how the brush
1:23:38: kind of like Works under the hood but the brush doesn't need to be the only thing you know it's uh it's literally
1:23:44: just adding elements to the array but like once you have like you know collection support and Proto you can you
1:23:50: can you know do stuff with this all right like you'll be able to like you know for example um you know just use
1:23:57: this mesh and like proced generate whatever visuals you want you know like you want to generate like a spiral so
1:24:03: just make algorithm that just adds a point to make a spiral and you'll get a spiral out of this you'll be able to
1:24:09: make your own PR Sy them because you can just you know use the raw data tool and
1:24:14: make your own brush um you know like like just completely bypass the system if you want
1:24:20: if you want to or you could make tools that you know for example take this and do some processing to it maybe you know
1:24:26: they will smooth it out maybe they'll like you know you process it maybe you like add like you know like you'll make
1:24:31: like a processing thing that just iterates over all the points and you know does a thing where it's like you
1:24:37: know does something to each of the points so you'll be able to make tools like that but also like you know it's a
1:24:43: it's a prerequisite for lots of other functionality I'm going to move back
1:24:49: [Music] now hello
1:24:55: here we go you know there's um there's lot of other functionality like um like
1:25:00: one that comes very often Jason parsing because um when you parse Jason you
1:25:06: essentially get a collection you get you know like a dictionary of you know tokens like you know and keys and then
1:25:13: like you know those can be for the collections you know uh so then on itself I think it's going to open up a
1:25:20: lot of cool options other stuff like right now we have the r one un Noe so you can like you know make a rast but
1:25:26: like it it it gives you whatever is the first which makes things difficult because sometimes you want to filter
1:25:31: things um so it's um
1:25:37: it's what's the word um like it's sort of like a Gateway for
1:25:43: lots of other features like it's going to open like it's not only going to like create like possibility for lots of cool
1:25:50: creations but also it's going to open up doors for lots of features that we're not able to do properly you know until
1:25:57: that's in place so once we have it in We'll add like a Json note like you just
1:26:02: plug Json and you get you know pass output and it can just enumerate the results same way you know opposite you
1:26:07: construct a collection of things and you'll be like two Json and it's going to make like like interrup way easier
1:26:14: with the RAS note that I was mentioning earlier um you'll be able to get a list of all hits and you can just iterate
1:26:20: over them find the hit you want you know and you have like a lot more flexib that way um it's much same thing like you
1:26:28: know like for example working with slots or once we add component access that also gives you collections like lots of
1:26:35: things they're essentially collections um and once you have the native support is going to be able to
1:26:41: interact with those so I think that is like one of those things that's going to
1:26:47: just explode things and also in general like if you're making your own code often times collections can be a really
1:26:53: good way to make code like you know make like Implement certain things because you work on collections of things and
1:26:59: internally even if those collections don't go into the data model um so it's
1:27:04: just going to make you know programming a lot of things like way easier than it's now and more performant so yes I
1:27:10: think that that's going to be one of those things that's going to like um it's going to be one of those things
1:27:16: that just going to blow up fuzzy bipolar bear is asking FRS
1:27:23: there night that one I mean this now that's kind
1:27:30: ofir maybe maybe could consider this I don't
1:27:35: know um Al let me check on time half an hour left so think we're bring pretty
1:27:44: okay uh next question elector spy vstd plugin support one w i mean that one so
1:27:52: is the thing I've been kind of thinking about like you know with the uh prolex DSP for audio the one
1:27:59: problem well there's multiple problems but like one that I see is compatibility or like Arizona is
1:28:06: designed to become a multiplatform vstd I don't think they are like there might be like you a window specific and so on
1:28:13: the other one is security because um when I have like running vssp like it can be like an arbitrary code and there
1:28:20: is for you know that prolex DSP to work for for everyone unless you're rendering
1:28:26: it on just par user and then syncing the resulting audio stream everybody will need to run their code which means do
1:28:33: you synchronize the VST what if it's malicious you know like it's not safe to do you know that would open a vector for
1:28:40: attacks um so the only way I see that working is
1:28:46: you know like if you run it in a mode where only single user is actually rendering locally and then streaming audio but it doesn't fit while into
1:28:53: resonate where like you know everything's kind of synchronized and everything can be also persisted like so like because if you have a VST module
1:29:00: and you save your thing and then you load it on a different computer where you you don't have that vstd do we save
1:29:06: it together like you know do we save it and then just install it on your system like um and that opens up you know the
1:29:13: whole thing what if somebody else spawns it and there malicious code you know and I like we are malicious code on there
1:29:18: and it makes things kind of complicated in that sense so maybe maybe like some
1:29:25: way like limited way to make it work but it's trickier maybe if there's like a
1:29:31: way to work with the assembly that would open up like make it safe to do that kind of stuff but I don't know enough
1:29:37: right now oh there we go again oh I'm going to turn it I'm going to turn it
1:29:44: off on a second thir this might not have been the best
1:29:49: um this might not have been the best i f
1:29:55: um but yes uh let me know it's like but yeah like maybe like I don't
1:30:02: did I didn't do like enough research right now to like you know uh
1:30:09: um they like you know like give you like a super clear answer Alo s's asking I am outside your door can I come in the room
1:30:16: and say hi I mean you can but I don't know if you're actually watching SRA
1:30:21: okay how long have you been waiting
1:30:27: thing um you need to go closer is
1:30:33: not I can smell your breath did did you hear
1:30:40: s s
1:30:48: s oh no s s over slips so like you just you just you get um you get back
1:30:55: background [Music] noise I do it on my own my voice is L
1:31:03: I'm actually I have soda how I have soda I'm going to grab soda so there's like little intermission
1:31:21: SRA so there's about 20 minute lag on the quest questions I guess I mean
1:31:27: setty that's quite quite a bit of
1:31:33: persistence so yes um I hope that answers your question SRA who's not
1:31:39: here he he he did he answer his own question or I answer his question and
1:31:46: then I don't know I I don't know what I'm saying but yeah that's that's that's that is that question resolved
1:31:54: uh so next one is from cyob yoru would marching cubes tering be possible in the
1:31:59: future yes I mean um one of the previous resonance episodes um I've done like a
1:32:09: bit explainer on like the tering system so I don't want to go too much into detail on that one right now uh there's
1:32:15: a video on the official YouTube channel uh I recommend watching it I go into fair amount of detail you know on how
1:32:21: the Teran system is going to work and so on so uh give that one sorry give that one a
1:32:27: watch um but pretty much yes um what actually want to do so um sorry I Dr
1:32:35: soda and I'm full of bubbles um what I want to do is just generally integrate
1:32:42: the marching cubes algorithm um into our own like you know classes so we can run it on any data
1:32:49: because one of the cool things you know like once you have that algorithm there's lots of features you can build
1:32:55: with it one of them is for example you know metabol uh support uh because
1:33:00: usually that's you know rendered like it convert to a mes using marching cubes algorithm um and what we could do and
1:33:08: what I would really want to do is add a renderer module for the new particle system for Photon dust where each
1:33:16: particle is like you know like a metall and they actually you know instead of like rendering asual
1:33:22: particles it generates a mesh because that can look like a liquid so you could use it you know for some sort of a
1:33:28: simple liquid simulation uh so having um having you know
1:33:39: like having having like you know that algorithm sort of in the library we can
1:33:45: just you know run the whole thing run the particles through it uh you know generate like the same distance feeli
1:33:51: and you know convert into a mes and now we can do like fake liquids with a
1:33:56: particle system with Photon dust uh so having that you
1:34:02: know it's going to open a of options so I've been actually looking at like the
1:34:07: implementation and possibly adting one because I want to do that one soonish if
1:34:13: there's time like one of the fun things for Photon T but I'll see like no premises at this time uh barable fre
1:34:20: alarm clock yeah I'm I'm not I'm not a good alarm clock like I wasn't sure what s was doing because we we're staying at
1:34:26: a friend's house and um um uh and like there's stuff going on so
1:34:35: like there's a l of around of stories I don't know I sent him a message because
1:34:41: I wasn't sure like he might have been like working on some of his own stuff and such but been like working on the um
1:34:50: like getting like the V working on Linux getting his nice in there um so next question Shadow X uh
1:34:59: when ging spotting is done will you start on the audio system right away or is there more particle work and under
1:35:04: project uh there's still more particle work like Photon does is still considered kind of in experimental phas
1:35:11: um we need to get it like you know polished and out I know there's a spch of compatibility issues that I need to look into uh people have been making
1:35:18: reports um so once I'm kind of done with the know fun thing uh I'll Circle back
1:35:24: the dose uh fix up the issues uh we want to bring it out of experimental um and
1:35:30: essentially have it like a replace you know the particle system so once those issues are resolved uh Photon dust is
1:35:36: going to come out of experimental and the old particle system the Legacy one is going to be removed completely uh
1:35:43: making you know Photon do essentially like you know the the only particle system we have um so but we're not going to you
1:35:52: know do that until like we resolve like majority of like the issues like once we kind of confident it can you know
1:35:58: preserve existing content and it serves as a good kind of replacement so there's still work to do on that one um there's
1:36:07: uh I don't know what like in in between projects I I'll see like how things going to go um so but the next next
1:36:17: bigger thing that's going to for part of the big performance update uh is essentially know it's going to be the
1:36:22: audio system and also is asking isn't S already working audio uh SRO yes he's
1:36:28: been he's working on a part of it uh he's been specifically working on the Reverb Library uh because the audio
1:36:35: system the one we have right now you can have Reverb zones and we need to preserve those in some way the problem
1:36:41: is you know their Reverb zones they're specific the unity specific with mod and
1:36:47: we're not going to be using that which means we need to provide an alternative um so s's been integrating uh this
1:36:54: Library called the ZB uh which is part of Library called sound pipe and making sure you know it
1:36:59: works with res uh so that part of work like you know he's already like is already Prett much done it's going to
1:37:06: make you know the parts that I work on then like I don't have to worry about it part and say me uh quite a good amount
1:37:12: of time uh so should answer those questions so
1:37:20: with that um we have about 23 minutes left 22 minutes left um there's no more
1:37:27: questions right now so if you want like any questions uh make sure um make sure
1:37:32: not can ask them um before you at Tim RS out like last time like we got some questions
1:37:38: like the last minute and at that point like it's too late to answer them so um if you got any questions like get them
1:37:44: in you know sooner than later um but that like I might like you know end like
1:37:49: doing bit kind of rambling about stuff I could actually talk about bit of the SP thing too because um wonder if I have a
1:37:58: the video somewhere uh one of the things I can't
1:38:04: show you because I'm not running on the right build um but one of the things I'm actually adding for gasing
1:38:10: splatting um are um spherical harmonics
1:38:15: and like I was mentioning earlier gan gan suppli they use spherical
1:38:22: harmonics uh I'm going to grab my brush over there um
1:38:27: let's go over there we Oh wrong [Music]
1:38:36: thing so spherical harmonics it's sort of like a way the encode directional
1:38:41: information like on a spe so like imagine like you know
1:38:46: imagine like you have like defined like kind of you have defined like you know like start and end and you can have like
1:38:54: a wave there and I'm just going to like here
1:39:00: and like you know this wave like you know you you need just a single parameter to kind of say you know how big this wave
1:39:07: is and then you know say you could also have wave that like you know goes in twice so it's for example like you know
1:39:14: one two and you also have parameter that says you know how big this wave is you
1:39:20: know so it can be different scales and these then you know composite on each other so what spherical harmonics is
1:39:28: it's sort of a way of doing this on the surface of aere essentially you know you have sort
1:39:34: of like almost like um the term for it like standing waves like on a surface of
1:39:40: a sphere so like if you like you know something like this imagine this is a sphere um you know imagine like you took like
1:39:48: this and you know wrapped it like this so you're going to have like in a wave it's going to be
1:39:54: like and then look I kind of throw it well this is going to be easier once
1:40:00: once I have these changes merge because it can just show you on the procedural objects but essentially you can have like you know the wave goes up and then
1:40:07: wave goes down and it kind of creates like you know this kind of like visuals so like
1:40:13: what it does if you if you mop this on a sphere and say for example say for example this inkles brightness what this
1:40:20: would do is make you know that like the more your you are here you know it's going to be brighter so like this part
1:40:26: is going to be really bright and it's going to be darker and it's going to be like the opposite you know there's going to be negative brightness so it's going
1:40:33: to be weak one and it's going to be strong one and then you can have like you know multiple levels of it so you
1:40:38: can have like a wave then you know it goes it sort of goes up goes down and
1:40:44: goes up and goes down and you know and drawing it very poorly but essenti have
1:40:49: like you know multiple kind of waves and then they kind of add together
1:40:55: um and what the end result of that is is that you can sample the information like
1:41:02: this mathematical representation can be used for information that changes based
1:41:08: on the direction uh so you can have a sphere and we have like information inod of
1:41:14: spherical harmonics that can be for example core um and you say you know I have like a viewer
1:41:21: here and I'm looking at the sphere from here so the direction is this so I'll
1:41:28: sample whatever whatever the wave you know is because it can be kind of
1:41:33: complicated you know with multiple of them I'll sample it and get over the colors here and then I'll look over here
1:41:40: if I have another viewer here and I look here you sample it this direction and I get a different value so like the
1:41:47: information changes based on the angle and this is useful for lots of things
1:41:52: one of them is you know it's a very efficient way uh to encode color information like light like ambient
1:41:58: lighting information is actually what un uses you know for the Ambiance of the world it encodes the information into
1:42:05: second order SP harmonics um and then samples it which is like a very
1:42:11: quick uh similar with the gassian splits each gassian it encl color using
1:42:16: spherical harmonics um and like you know that way it can
1:42:22: actually change color depending on the angle you do it um but it's also useful for lots of other things like is and the
1:42:29: way I've implemented into resonite is a general structure so you can encode any
1:42:34: any data type that can be supported like addition and multiplication you can encod it in spherical harmonics and then
1:42:40: you can actually sample it you know based on the direction so you'll be able to use it yourself you know in your own
1:42:46: Creations where you want you know you specify your harmonic and then you can sample just provide a direction and you
1:42:51: get whatever value um um there's also interesting and it's kind of funny
1:42:56: because like some like sometimes things end up having unexpected like um
1:43:03: unexpected I'm going to move back over there uh sometimes things have like unexpected like
1:43:10: synergies um I'm like earlier like one of the things for the particle system like I might end up doing is having to
1:43:16: compute shaders to generate things to get around um you know to get around some of
1:43:24: the like performance issues with phon does when submitting data and it's kind of similar mechanisms that are being
1:43:29: used for rendering the gasin plus because there's a lot of kind of like overlap so some of the work I'm doing now actually kind of helps helps with
1:43:36: that then also the spherical harmonics that helps like you know exposing some of the stuff and preserving some of the
1:43:42: stuff how unit it does light things so we can you know expose that but also when I'm once I'm working on the audio
1:43:48: system uh spherical harmonics they're used to encode ambis which is direct
1:43:55: audio added something that we had number of people like request and with that support already being added it actually
1:44:01: makes it trivial um to decode those because like literally with Amazon have
1:44:08: multiple channels and the channels are the coefficients for the spherical harmonics and we already have codes so
1:44:13: like you know sample those directionally um so funnily enough it makes it very
1:44:20: easy you know to support foric so we like as part of the audio reorg and we might just get Sonic support um just
1:44:27: because you know all of these kind of basic Primitives like they use in of different system
1:44:53: oh hello are we back oh sorry my the headset Wi-Fi died um I'm on the call
1:45:01: Quest can you hear me hello this one
1:45:08: to can you hear me fine okay sweet thank you yeah the headset Wi-Fi just
1:45:13: completely died um but yeah like like it's one of those things where I'm
1:45:20: like like I originally added it for this like thing that was just the
1:45:25: fun thing and I'm like now it's usedful for this this and this so like I'm like I'm I'm happy I like having more you
1:45:31: know General Primitives and BR engine that like can be used for lot of things
1:45:36: um but that jackx there is asking when will you make me hungry with the gassian food scans once is implemented very soon
1:45:43: like the moment is in the build I'm just importing my hundreds of cian plants and
1:45:50: eating everybody's V with them you know ding them people's faces to make them hungry I I did already do a bunch of
1:45:57: gassian splits of food and it looks like amazing so it's going to happen soon um nun is asking last time I asked
1:46:05: about using systems like a particle system has to build shaders from Parts like one part is an OB detection is
1:46:11: normal map and you can stack them it wasn't clear if you thought you could but my understanding is that these are
1:46:16: just pieces of code that you can stack what would be sure with that I I'm S I
1:46:22: don't I understand the question
1:46:29: um like paral system doesn't really have anything to do with building
1:46:36: shaders like I'll be like like you could use you know materials with it but like I
1:46:42: don't I'm sorry I don't like you have to like rephrase the question I don't really understand what you're
1:46:48: asking um next question Shadow X is asking in the future sure could there be
1:46:53: an eraser for geometry line brushes how expensive would it be distance to check every point in a stroke against eraser
1:47:00: yeah this one of things you could do like and I do would like to have us have more tools you know for working with The
1:47:05: Strokes so you can like do stuff like smoothing them and also erasing them um it's not like it kind of depends how
1:47:12: many you know points there are usually if there's like certain amount you can just kind of loop through all of them
1:47:17: and do distance check it's going to be fast enough especially if you Flo it like you know r or something
1:47:23: um if it's bigger one one thing you can do is build acceleration structure uh which will you like you
1:47:30: know more efficiently sample things in space that's things that for example physics engine does you know like physic
1:47:36: engine you know in this world it can be hundreds of colliders or thousands or
1:47:41: even more and it's not going to you know Loop through every single one of them Collision it uses acceleration structures uh usually one based on some
1:47:49: some form of three um that makes it uh
1:47:54: way more efficient to sample things so you could do that as well question is you know like are we going to have like
1:47:59: a thing that does it automtic CL that's going to happen you know when you bring the Eraser and stuff like that but it's
1:48:06: it's possible it's just a matter of you know investing time into implementing it um check the fox author also I do have a
1:48:13: question if there's still time when you get to it uh I'm currently building a little spaceship game currently resite
1:48:20: Creations like this can only be shared in resite plans allowing external St distribution of games made within
1:48:26: resonite um so we do have General ideas like we want to do it so like you can
1:48:32: like you know um you can essentially like say like this is a bunch of Worlds I've built export these as a standard
1:48:39: executable um so that's something we want to give as a tool to people it's very unlikely to happen while still on
1:48:46: Unity because I don't think it works with their licensing um maybe if you like you know generate a unit project
1:48:53: and you did compil with your own version of unity that would work um uh but once
1:48:59: we know like running on our own engine like have licensed everything so we can just plop out an exe and do it uh it
1:49:06: might there's like things also like we need to figure out like you know the business model of it that um so it might
1:49:12: be like a paid feature or something like that's going to kind of depend um we do
1:49:18: have currently already some level of support like uh we have like a wi label kind of like way where we can make
1:49:25: builds of resonated like branded or something else um but this mostly for like really big companies because it
1:49:31: does require some manual work but also like you know commercial so if something's going to happen there's not
1:49:37: too many details in the specifics yet uh so that's going to come back time um also thank you so much for the
1:49:44: subscription with prime jbox there and no worries and thank you for helping get
1:49:50: user set up um bable best Tha high arm warmers we'
1:49:57: gotten I I ordered Ang PA in
1:50:04: April and I'm still waiting on them because the shipping compan is really bad and they screwed up
1:50:11: and I I I'm I'm looking forward like I've been like looking forward those because they they have like ones that
1:50:16: are like my colors they're like yellow and orange and I hear like like the super
1:50:22: comit is super good but like I don't know when they're going to
1:50:29: arrive I I I've been I've been waiting
1:50:34: like how much is it like eight months at this point it's the the the shipping
1:50:40: compan is pretty bad um nun is asking
1:50:45: will I be able to use Cas spot Ms to take a scan of a section of my first suit fur and somehow tiled it on my
1:50:51: avatar it seems like it would be way totic for without deep simulation you can't really use it for
1:50:57: that um like it's like you get like M but you cannot really tile it easily and
1:51:02: I don't think it's a good representation like you know of it because um usually just render gas in spot by itself They
1:51:09: Don't Really blend well with with each other super much um and it's also like
1:51:15: you know it's it's going to be there's like multiple problems with that because like it's not going to
1:51:20: be like it's not going to move it's not going to you know respond to anything it's not going to respond to Lighting in
1:51:25: the world all the lighting is baked into it so like if you're in Dark World the first is going to be glowing um you know
1:51:34: it's uh like they're not useful for that kind of stuff they're useful for like you know showing things you've captured
1:51:39: you know in the real world um but I wouldn't use them for I wouldn't really use them for like
1:51:46: you know like representing like avatars and stuff like that I'm some kind of make ins SP Avatar is just you know
1:51:53: aratar but like they're not the best suited for that kind of
1:51:58: thing um next question pH bipolar bear amics
1:52:04: and the gas ins reminds me of the light field photography Stu from few years ago lighter if all these things are interes
1:52:10: linked and working on one will give used as others I mean there's some overlap
1:52:16: like some of the math is like you know like the the more basic math you use
1:52:22: like the more more than you know it has like if you consider something like linear algebra that's just used all the
1:52:28: way across like lots of things with computer Graphics you know game engines and so on stuff like you know complex
1:52:35: numbers quter um you know like trigonom that's used in so much stuff
1:52:42: that like you know having like more stuff there just opens up possibilities um so to some extent yes
1:52:50: you get like you know more um you get kind of like you
1:52:55: know like the more of The Primitives you kind of have in the engine and you your code it makes it easier to implement
1:53:02: those other things and sometimes they're kind of you know build from these kind of common building blocks but with some you still need you know to do extra work
1:53:09: so it kind of depends you know on some of them um it's um I think like gas and SPL
1:53:15: they're like more over like better than light Fields because light fields are very very
1:53:20: memory heavy um and Y in Sp are kind of like more efficient with that they're not as
1:53:26: efficient as 3D models but uh um you know it's a different kind of tradeoffs
1:53:32: kind of thing
1:53:37: um yeah we got about six minutes left so like I think I have time for like a few more questions if there's some more um I
1:53:45: might otherwise like play with some more Rambles um maybe like you know like one
1:53:51: or two questions depending how long they are so if you got like some like uh it's probably like your last chance um like I
1:53:58: would also like just generally maybe like ask like like everybody's been doing good like this year um it's been
1:54:05: you know this pretty much the last episode of this year um and I'm kind of
1:54:12: glad you know kind of started like I hope like everybody's been enjoying like these like their reson podcast and
1:54:17: learning more about resite um I'm literally like glad like I've got an opportunity to sort of like you know
1:54:24: um um I would like you know um like I kind of wanted to talk about L
1:54:32: of the things you know and sometimes didn't know just kind of people and rumbling there tonight but this kind of gives a way to shareed it like in over
1:54:39: the wider community and there's been lots of ideas that like I wanted to kind of know share and just kind of give you
1:54:44: overall vision of like where is the going what do we want to do with it you know when which direction is heading and
1:54:51: um just you a bit more like behind the scenes as well you know sort of thinking
1:54:57: that goes into things so thank you very much like you know everyone who's been watching these and um you know I hope
1:55:04: like make lots more that's Jak is asking would you play song G made with like all
1:55:09: of them well depends how many there are probably not all of them like I um like
1:55:16: every like things already like reach a point like where I cannot keep up with everything that there is on night so
1:55:22: um if there's going to be a lot of them then I I don't think they'll be able to
1:55:29: which is in a way nice problem to have because it's like you know lots of people are using it but also like you
1:55:35: don't get the experience everything so it's kind of like um we'll see like the first few I'll probably you know play be
1:55:42: like you know this is stuff that people made but like once there's a lot of them like I don't think I'll have the
1:55:48: time um yeah um been like enjoying like you know doing
1:55:54: like these res podcast and like you know cutting them like into like videos that people can watch people some sometime
1:56:00: enjoying like you know the especially when it's cutting into smaller videos on specific topics there like some of them
1:56:05: it's also kind of interesting because it it works a sort of like a rough gauge of like you know what
1:56:12: things are people interested in like for example the performance one that costs like lots of use uh thein thing actually
1:56:18: got quite a bit of use I think like it feels like a lot of people are interested in that
1:56:25: um it's kind of interesting way you know to kind of measure you know how much people are interested in particle things
1:56:32: and it just generally you know give people kind of bite-sized things to like be able to share you know and have like
1:56:38: sort of flex Central information like you know in one place on how things are going like for example with the performance you know there's been lots
1:56:44: of rumors and LS of kind of like unknown kind of FL thing around people are like you know why is particle system being
1:56:51: worked on you know as part of performance you know like um and having think this sort of visual explanation I
1:56:56: feel it's going to help you know get people better understand you know the processes and the reasoning you know and
1:57:03: why we do you know the things that we do um in more like visual way as well so
1:57:12: thank you very much you know for like you know the opportunities or like all these thank you everyone you know for like using your anite and for supporting
1:57:19: us uh and making making it you know so we can kind of keep going and make lots of cool stuff and also for like you know
1:57:26: just you know like being part of the community making lots of cool projects and you know using resonite you know as
1:57:32: your like whether it's like you know your home like or whether it's like you know work through or whether it's you
1:57:37: know play who like you know you have fun um it's always like you know makes me
1:57:42: happy like when people enjoy the platform or whatever kind of purpose they want to use it for or multiple of them like there's you know OB lots of
1:57:50: lots of things to do and I hope they will even more things in the future um got like 2 minutes left uh I
1:57:57: don't think like if there's any last minute question maybe you'll be able to answer but um at this point like um I
1:58:04: can of guarantee you'll be able to answer answer them but um let actually ask you like in general like what kind
1:58:10: of things would you like to see like you know the new year to have like any New Year's resolution and stuff like
1:58:16: that or just like a bunch of stuff like I do want to like I hope like I get to like work on like myself um also thank
1:58:24: you very much everyone in the chat um but
1:58:31: [Music] yes it's been like it's been a lot of
1:58:38: fun doing these and like I hope like to get like do a lot more of these like in over like next year as well like I'm
1:58:44: glad I finally like these streams like it's something I wanted to start for so
1:58:49: long and it just never got like around like you know getting everything together through it because there's just
1:58:55: so much stuff all the time but I'm kind of glad like I did and thank you very much you know for everyone like you know
1:59:02: for supporting me and supporting like this platform and um I think we have a lot of kind of cool cool things in the
1:59:08: future so uh but that that's the last minute so thank you very much um I hope
1:59:15: like everybody has like you know had a great great 2024 and you're going to have even better 2025
1:59:22: uh I think the Creator G are going to be hosting the New Year's Event like like they usually do so um I hope you know
1:59:29: this's going to be a lot of fun uh I'll be around like I'll try to pop in but I'll be around like you know with
1:59:34: uh um you know with like people like we have like bit event like a lot of gathering with there so nice people so I
1:59:41: don't know how much will be like around but uh we'll probably have like running on a project or something um see if I
1:59:47: can bring my like West bro with me um so it's a
1:59:53: it's been like you know pleasure like chatting with everyone answering your questions and you know just being general part of the community so thank
1:59:59: you very much thank you for supporting us thank you watching these streams and have a Happy New Year you know have a
2:00:05: happy 2025 oh hit the button
2:00:15: no actually wait I'm going to rate I'm going to let me see who's streaming anybody
2:00:23: streaming anybody streaming res is just Creator Jam
2:00:29: there's anybody who streams [Music] resonate no it's not just a creative G
2:00:37: right now so Crea G it is um if you stream resonate like especially around this time like you're going to get
2:00:43: raided so I do recommend um streaming I don't this time you want
2:00:50: to stream Creator chin I'm going to do CH hit the
2:00:57: button
2:01:03: ball oh is it wrong one oh I think I typed it in the wrong
2:01:08: thing uh come
2:01:15: on this is this is hard to do in VR come on press this thing there we
2:01:20: go get right ready
2:01:29: right great CH okay ra is
2:01:38: ready okay so thanks again everyone than thank you very much you know for watching and um uh we'll see you see you
2:01:46: next year bye