"I love Code Slow"

2022-05-14 10:36:01

Code Slow Sunday, 18 April 2021 Rust Game DevelopmentEver since I started learning Rust my intention has been to use it for writing games. In many ways it seems like the obvious choice but the restrictions around lifetimes and ownership can make it hard to apply usual game programming patterns. Rust is sufficiently different from all the other languages to require a different its own way to think about problems.To learn more about how best to use Rust for game development I decided to write a Windows version of an Android game I had made a couple of years ago in Java. Doing a rewrite meant that all my efforts were focussed on the programming problems rather than game design issues.I wanted to make sure that this is a 'proper' game in the sense that it deals with all the ancillary issues such as saving progress, adjusting settings etc. and is built in a way that in a way that could be used as a starting point for other, more complex games. I also wanted to avoid, as much as possible, usingCellorRefCellwhich can be really useful but can sometimes chip away at some of the advantages of Rust.CratesThere are a lot of game development libraries for Rust that provide a one stop solution for sound, graphics, UI, controls, etc. I think these can be great, especially in the early stages of development. My own preference is to assemble a set of set of smaller, more targeted crates. Rust's dependency management makes this particularly easy to do.Most Rust game development libraries push some kind of Entity-Component solution. I don't think there is anything wrong with them but I wanted to to 'discover' the need for one rather than assuming that I needed to use one. ( Also, having spent a lot of time debugging Guice on Java I have become weary of code that automagically wire up things behind the scenes )The downside with assembling your own set of crates is that sometimes they do not play nicely together and it can be tricky to find out why. I spent ages tracking down a problem caused by winit making drag and drop support the default which did not play well with cpal because they had incompatible Windows Thread Apartment models.The crates I ended up using are;glium. This is one of my favorite Rust crates. It is the almost perfect wrapper for OpenGL; it handles all the tedious and error prone state management but is close enough to the OpenGL API to keep my existing OpenGL knowledge relevantrodioIn my previous projects I usedcpalwhich is an excellent library but this time I wanted something more high level that would let me just focus on what to play and when to play it. rodio definitely delivered.glium-glyphThis crate does a nice job of making glium play nicely with glyph-brush making text rendering very easy in gliumnalgebra-glmI really like what is is trying to do and it works but ergonomics aren't always great. It never felt intuitive like the other crates I used and because it uses deep multi-level traits it plays poorly with rust-analyzerserde-jsonAnother crate that 'just works'High level architectureRust's ownership model puts strict limits on the what kind of architecture's are allowed. The biggest language restriction is that there can only be one mutable reference to a piece of data. Many design choices revolve around how to manage those mutable references. My experience is that the restrictions force you to think much harder about the design but this ultimately results in much cleaner designs.Below is the very high level design that I ended up with;PageThis is trait that each page object implements. Examples of pages are; start menu, settings menu, game page. Only one page is active at any time and the implementation of each page keeps track of its own mutable data. The pages communicate with the rest of the system by sending PageAction messages to the PageManager.PageManagerHolds mutable references to all pages and keeps track of the active page. The page manager routes calls to the active page and is also responsible for coordinating access to shared objects, such as configuration data or glium.PageActionPages send PageAction messages to Page Manager to accomplish tasks that affect objects that are not owned by the Page. Examples of PageActions are; Save Settings, Change Page, Change screen mode.When the game first starts up it first loads all the assets and then transfers the ownership to the PageManager.This design works quite well and makes the ownership of any data very clear. There is a danger thatPageManagercould evolve into agod objectas the project becomes more complex. I could foresee a situation where the number of PageActions could become hard to manage and tedious to maintain.I don't think the design ended up being that different from the Java version. The main difference is that it is much cleaner in Rust, mainly because the ownership rules make it harder to break your own design rules. There were times where it would have been short-term convenient for multiple pages to control the same mutable object but would have introduced longer term code debt (In many ways Rust makes it harder to create code debt)InefficienciesTheglium-glyphcrate sets glium's DrawParameters when a glyph_brush object is created. This means that if you need different DrawParameters you need to create a new glyph_brush object. Usually this would never happen but the game uses OpenGL scissor rectangles to constrain the draw area whichisset on DrawPrameters. The scissor rectangle depends on the window dimensions so whenever the window is resized the scissor rectangle must be adjusted. This meant that the entire glyph_brush object had to be recreated just to change one value.I had three ways to address this;Fork my own versionof the glium-glyph that let me override the DrawParameters. I didn't like this as the whole point of using crates is to outsource complexity.Submit a PRfor to allow DrawParam overrides. I liked this better but I wasn't convinced that solving my specific uses case would improve.Live with the inefficiencyAlthough recreating the entire object every time window size changes is horribly inefficient it doesn't really matter. It makes no perceptible difference to the user so there is little point in spending time changing it.Rust Enums are awesomeControlling all the different visual transitions can quickly turn the code into an unreadable mess. For example; when you enter a game page, it needs to control fading in, fly text in and out, drop the movable pieces and disable/enable the input.Before using Rust I would typically end up with tons of little variables that would either be used only during a particular sequence or have variables whose meaning depended on the current state. Both made it very hard to reason about the code and had a tendency to introduce some very tricky bugs.Rusts enums let me store the current state and the data relating to that state in the same enum. This stops polluting the code with state dependent variables. Combined with thematchfunctionality and its requirement for completeness enums are very incredibly powerful. The more I used them, to more uses I find for them.enum GameState{ ShowingNewLevel( f64 ), Playing, ShowingSolution( f64 ), InGameMenu, ChangingPage( PageAction, f64 )}Code readabilityIt takes a little bit time to appreciate this. Initially all Rust code is a bit overwhelming because there are so many new concepts and the syntax is unfamiliar. Once I got used to the syntax I found that it was actually faster to read and understand Rust code. Because Rust is so strict about object ownership it is easier to build a mental model about how the code works. (This is not entirely true for code that has complex lifetime annotations which I still struggle with)This project has been my pet project for some time with very infrequent opportunities to spend time on it. Usually when I don't spend time on pet project for a couple of weeks I find it tricky to get back into it. With this project I found it easier because Rust makes the state of any code more binary; if the code compiles I can be fairly confident that there are no unresolved design issues.ConclusionRust changes the way you think. Working with other languages after spending time with Rust feels like entering a wild west where anything goes.Not having well defined ownership or rules on mutability feels irresponsible.CodeYou can get the source code for the project ongithub. My original original Android game ishere.atApril 18, 2021No comments: Email ThisBlogThis!Share to TwitterShare to FacebookShare to PinterestLabels:game,rust,rust lang Sunday, 5 July 2020 Writing a winning 4K intro in RustI recently wrote my first 4K intro in Rust and released it at the Nova 2020 where it took first place in the new school intro competition. Writing a 4K intro is quite involved and requires you to master many different areas at the same time. Here I will focus on what I learned about making Rust code as small as possible.You can view the demo onyoutube, download the executable atpouetor get the source code fromgithubA 4K intro is a demo where the entire program ( including any data ) has two be 4096 bytes or less so it is important that the code is as space efficient as possible. Rust has a bit of a reputation for creating bloated executables so I wanted to find out if is possible to create very space efficient code with it.The setupThe entire intro is written in a combination of Rust and glsl. Glsl is used for rendering everything on screen but Rust does everything else; world creation, camera and object control, creating instruments and playing music etc.Some of the features I depend on, such as xargo, are not yet part of stable Rust so I use the nightly rust toolchain. To install and use the nightly toolchain as default you need the following rustup commands.rustup toolchain install nightlyrustup default nightlyI usecrinklerto compress the object file generated by the rust compiler.I also usedshader minifierfor pre-processing theglslshader to make it smaller and more crinkler friendly. The shader minifier doesn't support output into.rsfiles so I ended up using its raw output and manually copying it into myshader.rsfile. (In hindsight, I should have written something to automate that stage. Or even created a PR for shader minifier)The starting point was the proof of concept code I developed earlier (https://www.codeslow.com/2020/01/writing-4k-intro-in-rust.html) which I thought was pretty lean at the time. That article also goes into but more detail about setting up thetomlfile and how to use xargo for compiling tiny executable.Optimizing the design for code sizeMany of the most effective size optimizations have nothing to do with clever hacks but are the result of rethinking the design.My initial design had one part of the code creating the world, including placing the spheres and another part was responsible for moving the spheres. At some point I realized that the sphere placement and sphere moving code were doing very similar things and I could merge them into one sightly more complicated function that did both. Unfortunately, this type of optimization can make the code less elegant and readable.Looking at the assembly codeAt some point you have to look at the compiled assembly code to understand what the code gets compiled into and what size optimizations are worth it. The Rust compiler has a very useful option,--emit=asmfor outputting assembler code. The following command creates a.sassembly file;xargo rustc --release --target i686-pc-windows-msvc -- --emit=asmIt is not necessary to be an expert in assembler to benefit from studying the assembler output but it definitely helps to have a basic understanding assembler syntax. The release version usesopt-level = "zwhich causes the compiler to optimize for the smallest possible size. This can make it a bit tricky to work out which part of the assembly code corresponds to which part of the Rust code.I discovered that the Rust compiler can be surprisingly good at minimizing code; getting rid of unused code and unnecessary parameters and folding code. It can also do some strange things which is why it is essential to occasionally study the resulting assembly code.Using cargo featuresI worked with two versions of the code; one version does logging and allows the viewer to manipulate the camera which is used for creating interesting camera paths. Rust allows you to definefeaturesthat you can use to optionally include bits of functionality. Thetomlfile has a[features]section that lets you declare the available features and their dependencies. My 4K intro has the following section in thetomlfile;[features]logger = []fullscreen = []Neither of the optional features has dependencies so they effectively work as being conditional compilation flags. The conditional blocks of code are preceded by#[cfg(feature)]statement. Using features in itself does not make the code smaller but it makes development process much nicer when you easily switch between different feature sets. #[cfg(feature = "fullscreen")] { // This code is compiled only if the full screen feature has been selected } #[cfg(not(feature = "fullscreen"))] { // This code is compiled only if the full screen feature has not been selected }Having inspected the compiled code I am certain that only the selected features get included in the compiled code.One of the main uses offeatureswas to enable logging and error checking for the debug build. The code loading and compiling the glsl shader failed frequently and without useful error messages it would have been extremely painful to find the problems.using get_uncheckedWhen putting code inside anunsafe{}block I sort of assumed that all safety checks would be disabled within this block but this is not true, all the usual checks are still applied and these checks can be expensive.By default Rust range checks all array accesses. Take the following Rust code delay_counter = sequence[ play_pos ];Before doing the table look up the compiler would insert code that checks that play_pos is not indexing past the end of sequence and panic if that was the case. This adds considerable size to the code as there can be a lot of table look-ups like this.Converting the above code into delay_counter = *sequence.get_unchecked( play_pos );tells the compiler to not perform any range checks and just do the table look-up. This is clearly a potentially dangerous operation and can thus only be performed within anunsafecode blockMaking loops space efficient.Initially all my loops used the idiomatic rust way of doing loops, using thefor x in 0..10syntax which I just assumed would be compiled into tightest possible loop. Surprisingly, this was not the case. The simplest case;for x in 0..10 { // do code}would get translated into assembly code that did the following; setup loop variableloop: check for loop condition if loop finished, jump to end // do code inside loop unconditionally jump to loopend:whereas if did the following rust codelet x = 0;loop{ // do code x += 1; if i == 10 { break; }}would get directly compiled into; setup loop variableloop: // do code inside loop check for loop condition if loop not finished, jump to loopend:Note that the loop condition is checked at the end of each loop which makes the unconditional jump unnecessary. This is small space saving for one loop but they do add up when there are 30 loops in the program.The other, much harder to understand, problem with the idiomatic Rust loop is that in some cases it the compiler would add some additional iterator setup code that really bloated the code. I never fully understood what triggered this additional iterator setup as it was always trivial to replace thefor {}constructs with aloop{}construct.Using vector instructionsI spent a lot of time optimizing theglslcode and one of the best class of optimizations ( which also usually made the code run faster) was to operate on an entire vector at a time instead of operating at a component at a time.For example, the ray tracing code use a fastgrid traversal algorithmto check which parts of the map each ray visits. The original algorithm considers each axis separately but it is possible to rewrite the algorithm so it considers all axes at the same time and does not need any branches. Rust doesn't really have a native vector type like glsl but you can use intrinsics to tell it to use SIMD instructions.To use intrinsics I would convert the following code global_spheres[ CAMERA_ROT_IDX ][ 0 ] += camera_rot_speed[ 0 ]*camera_speed; global_spheres[ CAMERA_ROT_IDX ][ 1 ] += camera_rot_speed[ 1 ]*camera_speed; global_spheres[ CAMERA_ROT_IDX ][ 2 ] += camera_rot_speed[ 2 ]*camera_speed;into let mut dst:x86::__m128 = core::arch::x86::_mm_load_ps(global_spheres[ CAMERA_ROT_IDX ].as_mut_ptr()); let mut src:x86::__m128 = core::arch::x86::_mm_load_ps(camera_rot_speed.as_mut_ptr()); dst = core::arch::x86::_mm_add_ps( dst, src); core::arch::x86::_mm_store_ss( (&mut global_spheres[ CAMERA_ROT_IDX ]).as_mut_ptr(), dst );which would be quite a bit smaller ( but a lot less readable ). Sadly, for some reason this broke the debug build while working perfectly on the release build. Clearly, this is a problem with my intrinsics knowledge and not a problem with Rust. This is something I would spend more time on for my next 4K intro as the space saving were significant.Using OpenGLThere are lot of standard Rust crates for loading OpenGL functions but by default they all load a very large set of OpenGL functions. Each loaded function takes up some space because the loader has to know its name. Crinkler does a very good job of compressing this kind of code but it is not able to completely get rid of the overhead so I had to create my own versiongl.rsthat only includes the OpenGL functions that are used in the code.ConclusionMy first objective was to write a competitive proper 4K intro to prove that language was suitable for scenarios where every single byte counts and you really need low level control. Typically this has been the sole domain of assembler and C. The secondary objective was to write it using idiomatic Rust as much possible.I think I was fairly successful on the first objective. At no point during the development did I feel that Rust was holding me back in any way or that I was sacrificing performance or capabilities because I was using Rust rather than C.I was less successful on the second objective. There is far too muchunsafecode that doesn't really need to be there.Unsafehas a corrupting effect; it is very easy to useunsafecode to quickly accomplish something (like using mutable statics) but once the unsafe code is there it begets more unsafe code and suddenly it is everywhere. In the future I think I would be far more cautious about using unsafe and only use it when there really is no alternative.atJuly 05, 20209 comments: Email ThisBlogThis!Share to TwitterShare to FacebookShare to PinterestLabels:4k intro,opengl,rust,rust lang Older PostsHomeSubscribe to:Posts (Atom)Rust Game Development  Ever since I started learning Rust my intention has been to use it for writing games. In many ways it seems like the obvious choice but th... .image { width: 100%; } Writing a winning 4K intro in RustI recently wrote my first 4K intro in Rust and released it at the Nova 2020 where it took first place in the new school intro competition. W...Tiny Windows executable in Rust I have recently spent a lot of time writing pixel shaders and given that I have already written a pure Rust mod player I have started to t...Writing a 4K intro in Rust Some weeks ago I started looking at using Rust for making a 64K intro. I started out by making a  minimal window app that  takes up only 3....Search This BlogHomeAbout MeJani PeltonenView my complete profileReport AbuseLabels4k introamigacargochannelcratecrinklerdocumentationgamegliumglutinhoundmodnalgebra-glmopenglrustrust langserdesudokutestthreadBlog Archive ▼  2021(1) ▼  April 2021(1)Rust Game Development ►  2020(2) ►  July 2020(1) ►  January 2020(1) ►  2019(12) ►  December 2019(1) ►  July 2019(1) ►  June 2019(1) ►  April 2019(2) ►  March 2019(1) ►  February 2019(2) ►  January 2019(4)Simple theme. Powered by Blogger.