8.3.2.49. v4.7.10ΒΆ

commit e163e108792cd59d511d66b7ff7c136787735e5a
Author: Victor Lowther <victor.lowther@gmail.com>
Date:   Wed Nov 3 14:30:13 2021 -0500

    fix(transact): Antagonize the perf tests even more.

    Add content pack swapping as part of the job create and fail stress
    test, and fix a bug on how and when we mark jobs as not current that
    exposes that can cause occasional spurious job create failures.

M   backend/jobs.go
M   frontend/job_create.go
M   perf_tests/machineCreate_test.go

commit 243873fc1792552741bac7d1928d17790945cb1b
Author: Victor Lowther <victor.lowther@gmail.com>
Date:   Thu Oct 28 20:01:11 2021 -0500

    feat(transact): Refactor Transactions and fix resource leaks

    The API presented to the rest of dr-provision made coding for access
    to the transactions directly unweildy, and hid a couple of resource
    leaks that only show up under significant contention.

    1. Refactor the transact package to shift around how its functionality
    is exposed.  In particular, the transaction IDs themselves are hidden
    while the transactions themselves are exposed, and have methods that
    handle the heavy lifting.  This also adds the methods to panic with an
    appropriate TxError if lock acquisition fails, since that is the most
    common use of this package by far.  Everything else that uses
    transactions has been refactored to reflect these changes.

    2. It is possible under very high contention mixed with cancellations
    for a goroutine handling a transaction to exit without releasing the
    interlock tracker mutex.  Mutex management in the transaction
    interlock tracker has been refactored to use the with() pattern we use
    elsewhere in the codebase, and all functions that need to use the
    mutex have been refactored to account for these changes.  This
    resulted in splitting transact.Tx.waitOn into two smaller functions.

    3. Cancelled transactions that were waiting to acquire a write lock on
    something were not properly unblocking the key they were waiting on.
    The transaction release codepaths have been refactored to account for
    this situation and properly release the write blocks.

    4. The perf tests now batch up API requests to force maximum parallelism
    and place more stress on the anti-starvation code and the transact
    code.

    Do not run the perf test until you have a suitably high ulimit -n.  I
    recommend setting it to at least 100,000 open files.

M   backend/dataTracker.go
M   backend/dhcpUtils.go
M   backend/endpoint_manager.go
M   backend/jobs.go
M   backend/pluginProviders.go
M   backend/profiles_test.go
M   backend/requestTracker.go
M   backend/subnet.go
A   cmod/contentChange.go
M   datastack/stack.go
M   midlayer/dhcp.go
M   perf_tests/machineCreate_test.go
M   test/serverGroup.go
A   transact/interlockTracker.go
A   transact/simpleKey.go
M   transact/tx.go
M   transact/tx_test.go

commit 8c1104d058b4b3375c3490716106ed61257cdedd
Author: Zaheena <zaheena@gmail.com>
Date:   Tue Nov 2 13:12:34 2021 -0400

    fix(server): one less place for AllLocked. this was causing hung goroutines

M   server/args.go

commit 91274b46dc310f6905912155a4438a886f49f9ab
Author: Victor Lowther <victor.lowther@gmail.com>
Date:   Tue Oct 26 09:05:47 2021 -0500

    fix(mgrtests): Add missed updates to manager tests.

    All that prefs listing needed to have the purgeIdleDelay pref added.

M   manager-tests/test-data/output/TestAddMachines/client1.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/client2.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/manager1.prefs.list.2/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/manager1.prefs.list/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/manager1.prefs.set.manager.true.2/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/manager1.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/sub2-client1.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/sub2-client2.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/sub3-client1.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/sub3-client2.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/sub4-client1.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestAddMachines/sub4-client2.prefs.set.manager.true/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/client1.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/client1.prefs.list/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/client2.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/client2.prefs.list/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/manager1.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/manager1.prefs.list/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/sub2-client1.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/sub2-client2.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/sub3-client1.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/sub3-client2.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/sub4-client1.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestManagerFunctions/sub4-client2.prefs.list.e5b1e4bcad9ef0a7b91360e3033c60b9/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/client1.prefs.set.baseTokenSecret.01234567890123456789012345621091.systemGrantorSecret.01234567890123456789012345621091/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/client2.prefs.set.baseTokenSecret.01234567890123456789012345622091.systemGrantorSecret.01234567890123456789012345622091/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/manager1.prefs.set.baseTokenSecret.01234567890123456789012345611091.systemGrantorSecret.01234567890123456789012345611091/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/sub2-client1.prefs.set.baseTokenSecret.01234567890123456789012345631091.systemGrantorSecret.01234567890123456789012345631091/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/sub2-client2.prefs.set.baseTokenSecret.01234567890123456789012345632091.systemGrantorSecret.01234567890123456789012345632091/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/sub3-client1.prefs.set.baseTokenSecret.01234567890123456789012345641091.systemGrantorSecret.01234567890123456789012345641091/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/sub3-client2.prefs.set.baseTokenSecret.01234567890123456789012345642091.systemGrantorSecret.01234567890123456789012345642091/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/sub4-client1.prefs.set.baseTokenSecret.01234567890123456789012345651091.systemGrantorSecret.01234567890123456789012345651091/stdout.expect
M   manager-tests/test-data/output/TestSetupServers/sub4-client2.prefs.set.baseTokenSecret.01234567890123456789012345652091.systemGrantorSecret.01234567890123456789012345652091/stdout.expect

End of Note