Skip to content

fix: network clock adjustment overshoot and per-frame NetworkManager scene search - #4195

Open
NoelStephensUnity wants to merge 10 commits into
develop-3.x.xfrom
fix/clock-overshoot-and-unified-connections-scan
Open

NoelStephensUnity wants to merge 10 commits into
develop-3.x.xfrom
fix/clock-overshoot-and-unified-connections-scan

Conversation

@NoelStephensUnity

@NoelStephensUnity NoelStephensUnity commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Purpose of this PR

Three small, independent fixes.

Network clock adjustment overshoot: without this fix, a client's clock keeps alternating between running slightly fast and slightly slow on every frame, even after it has synchronized with the server. Interpolated objects then judder (a periodic motion artifact caused by mismatched frame rates): they move in uneven steps instead of smoothly. It is barely noticeable at the default NetworkTimeSystem.AdjustmentRatio, and clearly visible at higher values.

Per-frame NetworkManager scene search: without this fix, UnifiedUpdateConnections searches the whole scene for NetworkManager instances every frame, in every Entities world, even when there is no hybrid session, so the frame time grows with the size of the scene (about 1.5 ms per frame with 10,000 MonoBehaviours). It now only exists in client and server worlds, and only runs in a world created for a hybrid session, where it updates the connections of that world's own NetworkManager.

Hybrid mode in dedicated server builds: Netcode for Entities cannot create a single world host in a dedicated server build, which every hybrid server used. A dedicated server build now creates a server-only world, and a build for only one side logs an error when asked to start the other side.
(The last two are excluded from the changelog since the unified feature set has not yet been released)

Jira ticket

None.

Changelog

  • Fixed: Issue where a client's clock kept alternating between running slightly fast and slightly slow on every frame, even after it had synchronized with the server. Interpolated objects juddered (a periodic motion artifact caused by mismatched frame rates): they moved in uneven steps instead of smoothly. Barely noticeable at the default NetworkTimeSystem.AdjustmentRatio, and clearly visible at higher values.

Documentation

  • No documentation changes or additions were necessary.

Testing & QA (How your changes can be verified during release Playtest)

Functional Testing

Automated tests:

  • Covered by new automated tests: ClientNetworkTimeSystemTests.ConvergedClockAdvancesAtRealRate, UnifiedNetworkTransformTest.EachWorldUpdatesConnectionsForItsOwnNetworkManager, UnifiedUpdateConnectionsTests.NoWorldUpdatesConnectionsWithoutGhostPrefabs

Up-port

Not needed: this targets develop-3.x.x.

Backports

None.

…ing past it

Once a client's clock reached the offset it adjusts towards, each frame
stepped the full AdjustmentRatio past it and back, so LocalTime and
ServerTime alternated between running fast and slow every frame.
…r them every frame

UnifiedUpdateConnections called Object.FindObjectsByType<NetworkManager>() on every update, whose cost
grows with the number of MonoBehaviours in the scene: about 1.5 ms a call with 10,000 and 14 ms with
50,000. NetworkManager.Instances is filled in Awake and emptied in OnDestroy, and the system visits that.
@NoelStephensUnity
NoelStephensUnity requested a review from a team as a code owner October 16, 2026 15:55
@u-pr

u-pr Bot commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

⚠️ Some settings were ignored in '.pr_agent.toml (repository root)' repo settings

Check the configuration's TOML syntax, permitted settings, and size limit. See the error below.


Error message:
Ignored prohibited settings: github_app.handle_pr_actions. Permitted settings were applied.

Configuration content:
# This is the configuration for the u-pr bot
# https://github.cds.internal.unity3d.com/unity/u-pr
# For configuration of this file:
# https://developer.portal.internal.unity.com/catalog/default/component/u-pr/docs/configuration/automatic_runs/

[github_app]
handle_pr_actions = ['opened', 'ready_for_review']  # PR events that auto-run pr_commands
pr_commands       = ["/harness-review"]             # what runs on those events
reviewer_commands = ["/harness-review"]             # what runs when @u-pr is added as reviewer

@u-pr u-pr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Harness Review

LGTM

The changes address clock overshoot and repeated scene searches, with a remaining compatibility concern in manager discovery.

Reviewed commit 84cbb03

🤖 Helpful? 👍/👎

Comment thread com.unity.netcode.gameobjects/Runtime/Core/NetworkManager.cs Outdated
UnifiedUpdateConnections now runs only in client and server worlds that hold
the NetworkManager they were created for, and updates that NetworkManager
alone. This replaces the NetworkManager.Instances list, which a derived
NetworkManager declaring its own Awake never entered.
@codecov-github-com

codecov-github-com Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 5.76923% with 49 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ime/Components/Helpers/UnifiedUpdateConnections.cs 0.00% 40 Missing ⚠️
...netcode.gameobjects/Runtime/Core/NetworkManager.cs 0.00% 5 Missing ⚠️
...cts/Runtime/Components/Helpers/UnifiedBootstrap.cs 0.00% 4 Missing ⚠️
@@                Coverage Diff                @@
##           develop-3.x.x    #4195      +/-   ##
=================================================
- Coverage          76.11%   76.03%   -0.09%     
=================================================
  Files                159      159              
  Lines              27333    27345      +12     
=================================================
- Hits               20805    20791      -14     
- Misses              6528     6554      +26     
Flag Coverage Δ
NGOv2_project_testproject_ubuntu 75.09% <5.76%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...de.gameobjects/Runtime/Timing/NetworkTimeSystem.cs 95.09% <100.00%> (+0.04%) ⬆️
...cts/Runtime/Components/Helpers/UnifiedBootstrap.cs 0.00% <0.00%> (ø)
...netcode.gameobjects/Runtime/Core/NetworkManager.cs 72.37% <0.00%> (-0.28%) ⬇️
...ime/Components/Helpers/UnifiedUpdateConnections.cs 0.00% <0.00%> (-23.19%) ⬇️
Components Coverage Δ
com.unity.netcode.gameobjects 76.03% <5.76%> (-0.09%) ⬇️

ℹ️ Need help interpreting these results?

@NoelStephensUnity

Copy link
Copy Markdown
Member Author

@u-pr please review again

@u-pr

u-pr Bot commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Review Summary

The shift to binding the NetworkManager directly to the hybrid session's NetcodeWorld via the HybridNetworkManager singleton component is a great improvement. It cleanly resolves the previous subclass Awake inheritance limitation while avoiding scene-wide searches and cross-talk across multiple NetworkManager instances. The clock overshoot fix in NetworkTimeSystem and its accompanying tests are also well-designed.

A few observations and suggestions:


Observations & Suggestions

1. Null / Destroyed NetworkManager Safety in UnifiedUpdateConnections

  • Location: UnifiedUpdateConnections.cs:56, 94, 115
  • When using UnityObjectRef<NetworkManager>.Value, if the NetworkManager GameObject is destroyed prior to the ECS World being disposed (e.g., during scene transitions or test teardown), networkManager evaluates to null (or a destroyed Unity object).
  • Later dereferences (networkManager.IsServer at line 94 and networkManager.ShutdownInProgress at line 115) will throw a NullReferenceException or MissingReferenceException.
  • Suggestion: Treat networkManager == null as a disconnected/shutting down state:
    var networkManager = m_HybridNetworkManagerQuery.GetSingleton<HybridNetworkManager>().Value.Value;
    var isShuttingDownOrDisconnected = networkManager == null || networkManager.ShutdownInProgress || !networkManager.IsListening;
    (See inline comment for diff context).

2. Missing CHANGELOG Entry for UnifiedUpdateConnections

  • Location: CHANGELOG.md
  • CHANGELOG.md currently documents the clock adjustment/judder fix, but does not mention the UnifiedUpdateConnections performance improvement (replacing the per-frame FindObjectsByType<NetworkManager>() search with world-scoped manager binding).
  • Suggestion: Consider adding an entry under ### Fixed for this change as well.

3. Defensive Step Size in NetworkTimeSystem.Advance

  • Location: NetworkTimeSystem.cs:265
  • If an invalid or negative deltaTimeSec or AdjustmentRatio is ever passed, maximumStep would be negative, causing min > max in Math.Clamp(..., -maximumStep, maximumStep) which throws an ArgumentException.
  • Suggestion: Have you considered ensuring maximumStep is non-negative, for example:
    var maximumStep = Math.Max(0.0, deltaTimeSec * AdjustmentRatio);

🤖 Helpful? 👍/👎

…uilds

Netcode for Entities cannot create a single world host in a dedicated server
build, which every hybrid server used. A build for only one side now logs an
error when asked to start the side it does not support.

@EmandM EmandM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of nit-pick comments but this looks fantastic! I really, really like this approach!!

/// </summary>
internal struct HybridNetworkManager : IComponentData
{
public UnityObjectRef<NetworkManager> Value;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I LIKE this!!

Though a deep nit-pick, maybe Value isn't the best name here? It's very vague.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, Value didn't say much. Renamed it to NetworkManager, so it is now .NetworkManager.Value

m_CurrentLocalTimeOffset += deltaTimeSec * (m_DesiredLocalTimeOffset > m_CurrentLocalTimeOffset ? AdjustmentRatio : -AdjustmentRatio);
m_CurrentServerTimeOffset += deltaTimeSec * (m_DesiredServerTimeOffset > m_CurrentServerTimeOffset ? AdjustmentRatio : -AdjustmentRatio);
// Stop on the desired offset rather than step past it, or a converged clock alternates between running
// fast and slow by the whole ratio every frame, and everything rendered from it judders.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I don't understand this code comment. It wouldn't help me know what's going on here if I ever needed to interact with this code.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a fair point. The old comment described the symptom rather than what the code does. It now reads:

Move each offset toward its target by at most AdjustmentRatio of this frame's delta, stopping once it gets there. Stepping the full amount every frame overshot the target, so a synced clock alternated between running fast and slow, and anything driven by it juddered.

…updates

The hybrid world outlives its NetworkManager if ShutdownInternal throws before
disposing it, so UnifiedUpdateConnections now disconnects every connection and
announces none instead of throwing each frame. HybridNetworkManager.Value is
renamed to NetworkManager, the clock adjustment comment says what the clamp
does, and the unused Unity.Entities using that failed the Standards check is
removed.
@NoelStephensUnity

Copy link
Copy Markdown
Member Author

@u-pr please review again.

@NoelStephensUnity
NoelStephensUnity enabled auto-merge (squash) October 16, 2026 18:10
@u-pr

u-pr Bot commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Review Summary

The recent updates nicely address the previous feedback:

  • The isShuttingDown check (!networkManager || networkManager.ShutdownInProgress || !networkManager.IsListening) safely guards against dereferencing missing or destroyed NetworkManager instances.
  • Build-target differentiation in UnifiedBootstrap.CreateWorld properly configures dedicated server (CreateServerWorld) and client-only modes, avoiding invalid single-world host configurations in dedicated server builds.
  • The updated comments and renaming in HybridNetworkManager clarify intent.

A couple of minor observations and suggestions:


Suggestions

1. Repeated OnNetCodeDisconnect Invocations During Extended Shutdown

  • Location: UnifiedUpdateConnections.cs:119
  • If the world outlives the NetworkManager over several frames (e.g. if ShutdownInternal throws before world disposal), isShuttingDown remains true on subsequent frames. Because SystemAPI.Query<NetworkId>() matches all entities with NetworkId and does not remove NetworkId, OnNetCodeDisconnect will be repeatedly invoked every frame for the same entities.
  • Suggestion: Filter the query to only entities that currently have ConnectionState (which is removed in the first pass):
    foreach (var (networkId, entity) in SystemAPI.Query<NetworkId>().WithAll<ConnectionState>().WithEntityAccess())
    (See the inline comment for details).

2. Skip Discovery When Shutting Down

  • Location: UnifiedUpdateConnections.cs:78–88
  • When isShuttingDown is true, the pending connections loop (lines 89–111) is skipped, but new connections are still queried, allocated (new NetcodeConnection), and inserted into m_NewConnections.
  • Suggestion: Consider skipping the discovery query when isShuttingDown is true (e.g., guarding with if (!isShuttingDown)), or clearing m_NewConnections on shutdown.

3. Unused Local Variable

  • Location: UnifiedUpdateConnections.cs:54
  • var isServer = World.IsServer(); is not referenced within OnUpdate(). Consider removing it to keep the method clean.

🤖 Helpful? 👍/👎

// make sure we have disconnected all known connections.
if (isShuttingDown)
{
foreach (var (networkId, entity) in SystemAPI.Query<NetworkId>().WithEntityAccess())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the world outlives its NetworkManager across multiple frames while isShuttingDown is true, querying NetworkId without filtering will re-query these entities and invoke OnNetCodeDisconnect on every frame (since NetworkId is not removed from the entities).

Have you considered filtering the query with .WithAll<ConnectionState>()?

foreach (var (networkId, entity) in SystemAPI.Query<NetworkId>().WithAll<ConnectionState>().WithEntityAccess())

This ensures OnNetCodeDisconnect is invoked only once per active connection, as ConnectionState is removed in the first pass.

🤖 Helpful? 👍/👎

@NoelStephensUnity
NoelStephensUnity requested a review from a team as a code owner October 16, 2026 21:03
A client drains its incoming queue while it disconnects, after the unified transport has removed the
server connection, so a queued TimeSyncMessage made GetCurrentRtt throw KeyNotFoundException. It now
returns 0 for a connection it no longer has, as UnityTransport does.
… whether 6000.7.0b6 raised memory use

Replaces the b1.xlarge test. Revert before merge.
…ed test pass

The IL post-processor and compiler processes the editor starts stay resident through the test pass
and, on a b1.large agent, leave too little memory for the hybrid tests. The project is now imported
and compiled in its own batchmode session that exits first, any build process that outlives it is
stopped, and the job log lists the largest processes left before the tests start.

Also drops the temporary 6000.7.0b4 editor pin.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants