Chrono in Unreal, part 3: the traps

Part 2 described the architecture. This part is the list of things that went wrong anyway, sorted by topic, not by time. Every trap has a cause you can understand, and most of them could have been avoided with what we know now. That is the purpose of the list.

The demo the series is about. On YouTube in 1080p the console is readable.

The wheels

While floating, the left front wheel suddenly tipped to 50 degrees of toe-in. In the Chrono® state just as in the image, so not a presentation problem. There were two causes on top of each other.

The first: the paddle wheel thrust of the turning wheels acted along the X axis of the wheel spindle. The spindle turns with the wheel, so the thrust went around in a circle once per revolution and levered the knuckle over the dead centre of the tie rod. Get one sign or one axis wrong, and the force is no longer a force but a stirrer. The thrust direction is now the cross product of axle direction and vertical axis, independent of the wheel rotation.

The second: the Chrono vehicle has no rebound stop in the suspension and no steering stop at the knuckle, because the model is meant for driving on the ground, where neither is ever reached. An unloaded wheel in the water hangs fully extended, and in that position a small disturbance is enough to make the steering flip through. Both were retrofitted: a rebound stop as a limited spring force from 41 cm of extension, a steering stop as a limited torque around the kingpin from 45 degrees, with damping. The stop had to go from 35 to 45 degrees once, because the inner wheel in a corner already reaches 45 degrees at full lock. A stop that restricts normal driving is not a stop, it is a fault.

And one attempt in between, which was the most important lesson of the day: a water drag against the wheel rotation, as a quadratic torque on the spindle, made the simulation explode within seconds. The spindle is light, the torque was large, and at a 2 ms step size that is a stiff term the solver cannot hold. The rule ever since: own forces small and with an upper limit, torques on light bodies only capped, and every new force checked first in the debug start while floating.

Finally a reset crash: the accumulators through which own forces go onto Chrono bodies have indices. After recreating the system, the old indices pointed at bodies that no longer existed. Reset indices where the system is reset.

The automatic gearbox

The automatic gearbox in Chrono shifts according to the turbine speed behind the torque converter. On loose sand the wheels have heavy slip and dig themselves in. The converter sees almost no load but high input speed from the engine. The turbine never reaches the shift point, the engine hangs on the limiter at 2570 revolutions with little torque, the vehicle stays put and digs itself deeper. First solution: gearbox in manual mode, the pawn shifts according to engine speed.

Second trap, the other way round: if the wheels spin freely on the dune, the turbine spins with them, and shifting on engine speed puts it into second gear. There the reduction is missing, the turbine falls below the downshift point, and after 0.8 seconds of lockout first gear comes back. The log showed an oscillation between first and second gear every 0.8 seconds. Two rules ended it: upshift only if the turbine is turning at at least 90 percent of the shift point, because the next gear has 53 to 75 percent of the ratio, and only if the driving speed suits the next gear, 16 km/h for second, 30 for third. That is the kickdown behaviour of a real automatic.

Third: selector position 2. On the climb the autopilot limits to second gear, like a driver off-road. The first version switched the limit on according to the pitch angle and therefore also took effect on the jump ramp, where the vehicle is supposed to accelerate. It now applies by position, 25 metres past the foot of the dune until shortly before the crest. A rule that hangs on a measured quantity which also occurs elsewhere takes effect elsewhere too.

Third trap, only noticed on the second day: in the emergency stop the engine read 75 rpm in the HUD while the sound kept idling. The converter dragged the engine down with the locked wheels, and Chrono’s shaft engine has no idle governor; with the pedal up it simply spins down. A proportional throttle share under 900 rpm, at most 30 percent, holds the idle, and since the ABS keeps the wheels rolling the converter no longer gets into that state at all.

Triggers and time bases

The demo has three clocks: the world time of the engine, the simulation time of Chrono, which stands still during the start countdown, and the age of the drone, which also waits. Three clocks are two too many, but each has a reason, and every trap in this section comes from a trigger hanging on the wrong one or not being visible at all.

• The low pass of the drone was built as a variant of the orbit: smaller radius, less height, six seconds long. The radius is smoothed with a time constant of two seconds and never arrived at the bottom within six seconds. The log said “running”, the image said “nothing is happening”. Second version: explicit waypoints, 25 metres in front of the car and back close beside it. It ran in 4.7 seconds, 1.5 of them visible, and again nobody saw it. Third version: fly along one metre next to the car for six seconds, then away to the front. That was visible. And then the scene was moved to the end anyway, as a low pass back to the starting point with a stop on the rise. Four versions for one scene, because “trigger fires” and “viewer sees it” are two different things. Since then every trigger has a log line with start and end and is checked with screenshots at those times, not with the log alone.
• The long throw in the ball game is triggered when the vehicle is 90 metres away. The runner crosses the driving line around twelve seconds later. Those twelve seconds determine how fast the vehicle may arrive so that the emergency stop from 55 km/h ends up at the right place. Choreography is arithmetic with seconds.
• The water entry event for the sound is one edge per wheel: dry to wet, at most every 0.3 seconds. The surge onto the windscreen initially hung on the same edge with a minimum depth of 40 cm. On the edge, however, the wheel has only just become wet by 4 cm. The condition was never true. An edge and a level are two different events.
• Screenshots at fixed times are themselves a trigger: writing an image costs 200 to 400 ms and appears in the log as an outlier. Whoever measures performance has to know which outliers they produced themselves.

The autopilot

• The vehicle drove through the ball players, because the pedestrian guard was only active in the beach phase. It now applies in every phase, with a tighter loop, and after 0.8 seconds of standstill the vehicle reverses for three seconds.
• It drove into the same coastal rock three times. The rocks in the water were not in the obstacle list. Now they are, with the same evasion logic as the dune rocks, 3.4 metres beside the stone on the free side. That one rock also stands five metres further out.
• It got stuck on the dune when the small stones and the boulders were added: the traction control cut back to 70 percent from six degrees of slope, and with the slalom around the boulders the momentum was missing. Now 85 percent, and full when it drops below 15 km/h.
• It missed the jetty. One target point is not enough to get straight onto a 1.5 metre wide plank: now a turning point, then a lane follower on the jetty axis, full throttle only once the lateral error is below one metre and the heading angle below six degrees.
• At 55 km/h through more than five centimetres of water on saturated sand it did not get far. That is not a fault, that is the model. The choreography now drives on firm sand and only takes the wash margin for the braking zone.
• The drone flew through the mountain. The return flight followed the terrain below it, not the terrain ahead of it. Now it looks twelve metres ahead, climbs where the ground rises, and slows down there.

The camera

The chase camera stuttered over the washboard track. The first suspicion was the frame rate, the log said 17 ms mean, 20 ms maximum, no outliers. The cause was the ground clearance of the camera: a hard minimum distance, and every ground wave passing beneath the camera lifted it immediately and let it fall again a few frames later. The lift value is now smoothed, quickly up, slowly down. An earlier version of the same camera added the ground clearance again every frame, until it ended up in orbit. Two faults, one mechanism.

The drone camera kept showing a ring in the picture. The customer’s guess: the faked gimbal, and the drone has its own rotor in the picture. That is exactly what it was. The airframe tilts under the camera fixed to it, and on pitch or roll the translucent rotor disc swung into the frame from above. Two corrections: the drone no longer films itself, its own actor is hidden from its camera. And the gimbal now has end stops like a real one: it compensates the airframe tilt only up to 20 degrees of pitch and 15 degrees of roll, beyond that the picture tilts with the airframe. During the backflip the picture turns over once, and that is right.

The jetty

On the jetty it stuttered, not after the plunge but during the whole run-up. The log line showed the physics at 15 ms per frame instead of 8, the graphics card uninvolved. The jetty consisted of 38 planks, each its own rigid body with a collision box, and the tyres were triangle meshes with a few thousand triangles. Four meshes against 38 boxes, every step.

The customer’s suggestion was the right one: put something invisible underneath. The planks are now picture only. The driving happens on a single invisible fixed body with 18 long boxes along the profile, 4.5 metres where the jetty is nearly straight, 1.5 metres in the curved kick. And the tyres are cylinders instead of meshes. The soil model casts its rays against either just the same, sinkage and traction in the sand are unchanged, and the physics now sits at 7 ms. Visible geometry and collision geometry are two different things, and the second may be as coarse as the driving allows.

The engine

• TArray::Add(Array[i]) is an assert in Unreal® 5.8, because the element is moved when the array grows. Two crashes at startup while the author was watching. Copy first, then append.
• A self-created ChSystem has no gravity in Chrono 10. The default value is zero. The vehicle floats, and it looks like a fault in the coordinate conversion.
• Chrono and Unreal argue about macros such as check, PI and TEXT. A single header that saves the macros away and includes Chrono, everything else never includes directly.
• Vector parameters of a material are three-dimensional. Whoever computes with them using two-dimensional texture coordinates gets no result, but the grey default material and a line in the log. Mask beforehand.
• A render target that samples a material in 400 individual draw calls, each with its own material instance, costs 20 ms on the game thread. The water phases dropped to 24 frames per second while the graphics card was at 12 ms. Now only the large droplets get the lens, at most a hundred, the small ones a bundled texture.
• The Python API of the engine expects rotations as roll, pitch, yaw, unlike the C++ type. The sun was below the horizon, the scene was black.
• A level without Shader Model 6 and without the extended luminance range also renders black. Two causes for the same image.
• Starting the game from Git Bash does not work, the shell converts the map path into a Windows® path. PowerShell®.
• A soft dark spot on the roof was the Distance Field Ambient Occlusion, not a hole in the mesh.
• The commandlet for the materials hangs on the live coding connection after the last log entry and has to be terminated.
• The Windows Game Bar, triggered from a script, assigns the hotkey to the sending process and recorded the wrong window three times. OBS Studio® works as soon as the audio device is in the scene with its endpoint ID.
• Patch scripts that modify source code by text replacement must be idempotent. One of them duplicated the tick once.

The finale

The second half of the demo, night, parasail and drone as bodies, had traps of its own. The models are in part 6, part 7 and part 8; here only what went wrong.

• Night became day. The moon as a second atmosphere light without priority computed water, translucency and fog with itself and drew its disc as large as a sun. Fix: priority for the sun, small source size, exposure fixed at the daylight value, the lights make the night. Moon sphere at 40 km and stars at 8 km drifted against the sky and stood in front of the clouds; both pushed out to 200 km.
• The canopy hung upside down and flew backwards. The lift along the body’s vertical axis has no restoring moment; the weathervane torque additionally had the wrong sign. Two small changes at the point of application and the sign.
• The boat never became stuck, because the swell kept lifting and lowering it. The condition needs the speed over ground, not the position.
• The drone flew through jetty and hill, because it set its height from the ground beneath it instead of the ground ahead. It then swung, because every jump in the setpoint became a jump in the rotor forces; a follower with a braking profile and an acceleration limit smooths it. And it planned its departure from the sunk vehicle two metres under the water, until the minimum height was taken against the maximum of terrain and sea level. After the motors were cut the controller without thrust produced an invalid attitude; no controller without thrust, a guard restores the last valid pose.
• The sidelights shone 10 degrees too far aft, because a downward-tilted circular cone is wider in the horizontal than its half-angle. Amidships, horizontal, hard edge.
• The recording was silent three times. The audio device in the OBS® scene was invalid as an endpoint id after a headset change, the game without focus mutes its sound, and the software encoder stuttered. Default device in the scene, focus handed back cleanly, unfocused volume in the engine set to one, hardware encoder.
• Three build errors of the engine’s compiler were not errors in the model: a local variable shadowing an outer one, a console variable used before its declaration, a private property with Blueprint access.

What follows from this

Three rules that survived the day:

1. Every own force is formulated in Chrono coordinates, is small, has an upper limit and is checked first in the debug start that hits it hardest.
2. Every trigger has a log line with start and end, and it only counts as working once a screenshot at its time shows what was meant.
3. When it stutters, read the log first. The frame time says whether it is the engine, and if it is not, it is almost always a camera or a time base.

The sound, which comes about by other rules, is part 4, the models of the finale are part 6 and part 7.

Keywords: Project Chrono Unreal traps, steering stop multibody simulation, automatic gearbox simulation sand, selector position 2, trigger time base simulation, autopilot pedestrian detection, camera stutter ground clearance, Unreal Engine 5.8 plugin, serious gaming, simulation consulting Switzerland, swissEmbedded

Unreal® and Unreal Engine® are trademarks or registered trademarks of Epic Games, Inc. in the United States of America and elsewhere; these articles are neither endorsed nor reviewed by Epic Games. Project Chrono is an open-source project of the University of Wisconsin-Madison and Politecnico di Milano under the BSD-3 licence.