forked from Manuel/meeting-assistant
Observe owned diskutil stack and live StorageKit in Recovery
This commit is contained in:
dh
@@ -18,6 +18,8 @@ Full run 4191 at `9735db1` reached native Recovery: x86_64/root, Darwin 23.6.0 a
|
||||
|
||||
Run 4192 at `6122be2` captured the actual 603-byte guest file and strictly parsed version 14.6.1. Its host reply was published successfully, but the guest's reply-existence check timed out before services or disk enumeration. Guest request/timeout UTC timestamps were not retained, so late delivery and 9p visibility cannot be distinguished. The current candidate removes this version reply: the guest publishes its raw file and a provisional version candidate; the host's full XML validation and exact result binding remain mandatory before any installation permit. The later installation-permit transport is still unqualified.
|
||||
|
||||
Run 4193 at `9164fe4` successfully derived the same current-file version in both guest and host. Its single `diskutil list physical` process was stopped at 124 seconds by the 120-second watchdog without output. The observer saw one runnable row and 3.18 seconds of accumulated CPU time, without a stack or proven IPC endpoint. `ioreg` was also stopped before producing output. The post-disk service gates, permit, installation and tests were not reached. A prior modified Recovery14 reference image has byte-identical SystemVersion contents but a different image hash; it contains StorageKit with the `com.apple.storagekitd` and `com.apple.storagekitd.dm` MachServices, not `diskmanagementd`. The next observation targets that actual service family and the own diskutil stack; this reference does not establish the live service state of run 4193.
|
||||
|
||||
## Entry points and dependencies
|
||||
|
||||
Orchestration remains the .NET 10 file-based app `tools/ci/MacOsNativeDiagnostic.cs`. Existing Bash/Python boot integration is necessary before a guest SDK exists. NASM assembles the CPU probe in the disposable image build, without host/runner installation. No new runner, device, capability, secret or service is used.
|
||||
@@ -47,7 +49,7 @@ Native readiness requires x86_64, UID 0, macOS 14+, successful launchd service q
|
||||
|
||||
The raw file, exact source path, length, SHA256 and parsing receipt are retained and bound to the current token. This version evidence travels only from guest to host and requires no reply. The guest uses its existing Bash before any SDK exists; authoritative XML logic stays in C#/.NET. A Bash candidate alone cannot authorize installation or qualify readiness. This method establishes the current guest version, not successful execution of `sw_vers`.
|
||||
|
||||
Required commands retain 45 seconds, UID 180 seconds and the single disk query 120 seconds. The owned observer uses `/bin/ps -M -p <diskutil-child>` with a separate 60-second limit and two-second TERM/KILL grace. It avoids stack symbolication; thread waiting states do not identify an IPC endpoint. Observation failure passes no gate. Owned children are stopped on completion/cancellation; output remains 512 KiB per command and 4 MiB proof.
|
||||
Required commands retain 45 seconds and UID 180 seconds. The single disk query receives a 600-second diagnostic window under the existing 90-minute Recovery deadline. The owned observer captures StorageKit state, thread CPU snapshots and an optional one-second/100-ms `sample -mayDie` stack of only that live diskutil child. Each observation retains its own 60-second watchdog and two-second TERM/KILL grace. Missing tools, failed or timed-out samples remain explicit missing evidence; thread states alone do not identify an IPC endpoint. Stack output is captured directly from the owned state with a 512-KiB bound, without another native copy command. Observation failure passes no gate. Owned children are stopped on query completion/cancellation; output remains 512 KiB per command and 4 MiB proof. No service is started or restarted by the observer.
|
||||
|
||||
The container retains 6 GiB memory/swap, two-CPU limit, 512 MiB shared memory and a 4-GiB/two-vCPU guest. One fresh anonymous /storage volume holds the sparse 64-GiB target. Inspection rejects devices, capabilities, binds, ports, host networking and privileged mode. KVM is disabled with no /dev/kvm mapping; guest networking stays slirp.
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ static class NativeDiagnostic
|
||||
const string FullState = "/storage/14/ci-state";
|
||||
const string NativeVersionSource = "/System/Library/CoreServices/SystemVersion.plist";
|
||||
const string NativeVersionMethod = "guest-file/host-xml";
|
||||
const int MaximumDiskStackBytes = 512 * 1024;
|
||||
static readonly JsonSerializerOptions JsonOptions = new() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase, WriteIndented = true };
|
||||
const string OriginalBootstrap = "[ ! -e /tmp/m ]&&{ /sbin/mount_9p installstate >/dev/null 2>&1;exec /Volumes/installstate/launch.sh;};: >/tmp/m\n";
|
||||
const string MountOnlyBootstrap = "[ ! -e /tmp/m ]&& /sbin/mount_9p installstate >/dev/null 2>&1; : >/tmp/m\n";
|
||||
@@ -73,6 +74,7 @@ static class NativeDiagnostic
|
||||
{
|
||||
ValidateContracts();
|
||||
ValidateBootProgress();
|
||||
ValidateDiskStackCapture(output);
|
||||
await ValidateNativeSystemVersion(output);
|
||||
if (full) ValidateFullContracts();
|
||||
if (Option(args, "--source") is { } source)
|
||||
@@ -94,7 +96,11 @@ static class NativeDiagnostic
|
||||
nativeVersionMaximumBytes = 4096, nativeVersionResponseRequired = false,
|
||||
nativeVersionCandidateIsQualifiedReadiness = false, nativeVersionBindingRequiredBeforePermit = true,
|
||||
nativeVersionFixtures = "native-system-version-fixtures.json", nativeProductVersionCommandRemoved = true,
|
||||
diskReadinessAttemptLimit = 1, diskCommandLimitSeconds = 120, diskThreadObservationLimitSeconds = 60, stackSamplingUsed = false,
|
||||
diskReadinessAttemptLimit = 1, diskCommandLimitSeconds = 600, diskCommandExtendedForDiagnosticObservation = true,
|
||||
diskObservationCommandLimitSeconds = 60, stackSamplingRequested = true, stackSamplingDurationSeconds = 1,
|
||||
stackSamplingIntervalMilliseconds = 100, stackSamplingMayDie = true, stackSamplingRuntimeSucceeded = false,
|
||||
stackObservationIsQualifiedReadiness = false, diskStackMaximumCapturedBytes = MaximumDiskStackBytes,
|
||||
diskManagementDiagnosticLabel = "com.apple.storagekitd", diskStackCaptureFixtureCases = 4,
|
||||
resultNegativeCases = 6, containerNegativeCases = 11, recoveryPositiveCases = 4, recoveryNegativeCases = 12,
|
||||
independentFixtureCrc32Readback = true, resourceSnapshotRetention = true, cpuProfileSourceContractsVerified = true,
|
||||
preflightGateFixtureCases = 8, qemuRuntimePreflightExecuted = false, templateIsoDownloaded = false,
|
||||
@@ -279,6 +285,9 @@ static class NativeDiagnostic
|
||||
baseline = ReplaceOnce(baseline, "while (( attempt < 1 && SECONDS - readiness_start < 600 )); do", "while (( SECONDS - readiness_start < 600 )); do");
|
||||
if (Hash(Encoding.UTF8.GetBytes(baseline)) != "4d428f594dac14eff64ed87b172c81ecf85ac91da8c5460cd6ec4b1d310800c3")
|
||||
throw new InvalidOperationException("Outside seven explicit diagnostic blocks, two explicit native SystemVersion getter blocks and one-attempt limit, baseline identity/service/disk gates and watchdogs must remain identical.");
|
||||
foreach (var required in new[] { "command_limit=600; fi", "run_command management_before /bin/launchctl print system/com.apple.storagekitd", "observe_live_command storagekit-live /bin/launchctl print system/com.apple.storagekitd", "observe_command diskutil-stack /usr/bin/sample \"$disk_process\" 1 100 -mayDie -file \"$stack_output\"", "stack_output=\"$STATE_DIR/diskutil-stack.txt\"", "read -r -t 60 -u 9", "\"$BASH_VERSION\"" })
|
||||
if (!readiness.Contains(required, StringComparison.Ordinal)) throw new InvalidOperationException("Owned optional disk observation contract changed: " + required);
|
||||
if (readiness.Contains("PENDING_OUTPUTS+=(\"$stack_output\")", StringComparison.Ordinal)) throw new InvalidOperationException("Stack reports must be bounded directly by the Linux host, without another guest copy.");
|
||||
if (Hash(File.ReadAllBytes(Path.Combine("tools", "ci", "macos-native-bootstrap.sh"))) != "94f069e116fdc7685a4d233cab6fa50df9f39274386bb82157674061e74fadb5")
|
||||
throw new InvalidOperationException("Compatibility profile must preserve the baseline Apple recoveryosd wrapper.");
|
||||
if (Hash(Encoding.UTF8.GetBytes(OriginalDaemon13)) != "af9d7f6c1948079bd4384d27b6882678d6fb4e338fcf6a8be8f84fceef174ad6") throw new InvalidOperationException("macOS 13 allowlist bytes differ from the independently read comparison plist.");
|
||||
@@ -1158,6 +1167,7 @@ static class NativeDiagnostic
|
||||
ReportCpuPreflight(output, stage.ExitCode == 0 ? stage.Output : logs.Output);
|
||||
await Command("docker", ["exec", id, "head", "-c", "4096", "/run/shm/kernel-handoffs.log"], output, "capture-kernel-handoffs", cancellation, requireSuccess: false, retainSuccessful: true);
|
||||
if (token is not null) await CaptureNativeSystemVersion(id, output, token, full, cancellation);
|
||||
if (token is not null) await CaptureDiskStack(id, output, token, full, final, cancellation);
|
||||
var files = new List<(string, string)> { ("proof.log", "guest-proof.log"), ("result.json", "guest-result.json") };
|
||||
if (full) files.AddRange([("guest-phase.json", "guest-phase.json"), ("full-result.json", "full-result.json"), ("firstboot.log", "firstboot.log"), ("unattended-firstboot.log", "unattended-firstboot.log"), ("install.log", "install.log"), ("apple.log", "apple.log"), ("disk-ownership-ioreg.log", "disk-ownership-ioreg.log"), ("installed-root.plist", "installed-root.plist"), ("apfs-containers.plist", "apfs-containers.plist"), ("physical-store.plist", "physical-store.plist"), ("clt-catalog.log", "clt-catalog.log"), ("clt-install.log", "clt-install.log"), ("clt-sdk.log", "clt-sdk.log")]);
|
||||
foreach (var file in files)
|
||||
@@ -1182,6 +1192,55 @@ static class NativeDiagnostic
|
||||
await Command("docker", ["exec", id, "sh", "-c", "printf '[qemu]\n'; qemu-system-x86_64 --version | head -n 1; printf '[Recovery hash]\n'; test -f /storage/14/setup.dmg && sha256sum /storage/14/setup.dmg || exit 1; printf '[resources]\n'; df -Pk /storage; cat /sys/fs/cgroup/memory.max /sys/fs/cgroup/cpu.max 2>/dev/null || true"], output, "guest-container-resources", cancellation, requireSuccess: false, retainSuccessful: true);
|
||||
}
|
||||
|
||||
static async Task CaptureDiskStack(string id, string output, string token, bool full, bool final, CancellationToken cancellation)
|
||||
{
|
||||
var sourcePath = (full ? FullState : "/dev/shm/installstate") + "/diskutil-stack.txt";
|
||||
// The sample writes directly through 9p. Capture changing bytes with Linux
|
||||
// coreutils; absence/empty output must never suppress a later or final read.
|
||||
var capture = await Command("docker", ["exec", id, "bash", "-o", "pipefail", "-c", "head -c 524289 '" + sourcePath + "' | base64 -w 0"], output, "capture-diskutil-stack", cancellation, requireSuccess: false);
|
||||
if (capture.ExitCode != 0 || string.IsNullOrEmpty(capture.Output)) return;
|
||||
SaveDiskStackObservation(output, token, sourcePath, Convert.FromBase64String(capture.Output), final);
|
||||
}
|
||||
|
||||
static void SaveDiskStackObservation(string output, string token, string sourcePath, byte[] raw, bool final)
|
||||
{
|
||||
if (raw.Length == 0) return;
|
||||
if (raw.Length > MaximumDiskStackBytes + 1) throw new InvalidOperationException("Disk stack transport exceeded its one-byte sentinel.");
|
||||
var truncated = raw.Length > MaximumDiskStackBytes;
|
||||
var bounded = truncated ? raw[..MaximumDiskStackBytes] : raw;
|
||||
File.WriteAllBytes(Path.Combine(output, "diskutil-stack.txt"), bounded);
|
||||
Save(Path.Combine(output, "diskutil-stack-capture.json"), new { token, sourcePath, capturedBytes = bounded.Length, sha256 = Hash(bounded), maximumCapturedBytes = MaximumDiskStackBytes, truncated, finalCapture = final, snapshotOnly = true, sampleCompletionVerified = false, qualifiedReadiness = false, sampleExitAndTimeoutEvidence = "guest-proof.log", capturedUtc = DateTimeOffset.UtcNow });
|
||||
}
|
||||
|
||||
static void ValidateDiskStackCapture(string output)
|
||||
{
|
||||
var fixture = Path.Combine(output, "validation-disk-stack-capture");
|
||||
Directory.CreateDirectory(fixture);
|
||||
var cases = new List<object>();
|
||||
foreach (var size in new[] { 0, 64, MaximumDiskStackBytes, MaximumDiskStackBytes + 1 })
|
||||
{
|
||||
var folder = Path.Combine(fixture, size.ToString());
|
||||
Directory.CreateDirectory(folder);
|
||||
var raw = Enumerable.Repeat((byte)'s', size).ToArray();
|
||||
SaveDiskStackObservation(folder, "fixture", FullState + "/diskutil-stack.txt", raw, false);
|
||||
if (size == 0)
|
||||
{
|
||||
if (Directory.EnumerateFiles(folder).Any()) throw new InvalidOperationException("Empty stack capture was finalized.");
|
||||
}
|
||||
else
|
||||
{
|
||||
var retained = File.ReadAllBytes(Path.Combine(folder, "diskutil-stack.txt"));
|
||||
using var receipt = JsonDocument.Parse(File.ReadAllText(Path.Combine(folder, "diskutil-stack-capture.json")));
|
||||
if (retained.Length != Math.Min(size, MaximumDiskStackBytes) || !retained.AsSpan().SequenceEqual(raw.AsSpan(0, retained.Length))
|
||||
|| receipt.RootElement.GetProperty("truncated").GetBoolean() != (size > MaximumDiskStackBytes)
|
||||
|| receipt.RootElement.GetProperty("sampleCompletionVerified").GetBoolean() || receipt.RootElement.GetProperty("qualifiedReadiness").GetBoolean())
|
||||
throw new InvalidOperationException("Stack capture changed bytes/bounds or qualified an observation.");
|
||||
}
|
||||
cases.Add(new { size, success = true });
|
||||
}
|
||||
Save(Path.Combine(fixture, "validation.json"), new { success = true, cases, emptyCaptureFinalized = false, maximumCapturedBytes = MaximumDiskStackBytes, qualifiedReadiness = false, guestExecuted = false });
|
||||
}
|
||||
|
||||
static int NativeVersionRequestLength(string request, string token)
|
||||
{
|
||||
var fields = request.Split('\n');
|
||||
|
||||
@@ -142,10 +142,11 @@ read_native_system_version() {
|
||||
}
|
||||
# END native SystemVersion plist request helpers
|
||||
# BEGIN disk IPC diagnostic
|
||||
# Optional observations have their own child/timer ownership. Thread state/time
|
||||
# targets only this probe's diskutil and does not request stack symbolication.
|
||||
# Optional observations have their own child/timer ownership. Stack/thread
|
||||
# observations target only this probe's live diskutil; none qualifies readiness.
|
||||
observe_disk_query() {
|
||||
local disk_process="$1" output="$2" observation_child="" observation_timer=""
|
||||
local stack_output="$STATE_DIR/diskutil-stack.txt"
|
||||
cancel_observation() {
|
||||
trap '' TERM INT
|
||||
if [ -n "$observation_child" ]; then
|
||||
@@ -179,19 +180,38 @@ observe_disk_query() {
|
||||
kill -TERM "$observation_timer" 2>/dev/null || :
|
||||
wait "$observation_timer" 2>/dev/null || :
|
||||
printf '[disk-observation-exit] %s status=%s elapsed=%ss\n' "$name" "$status" "$((SECONDS - started))" >> "$output"
|
||||
OBSERVATION_EXIT="$status"
|
||||
observation_child=""; observation_timer=""
|
||||
}
|
||||
observe_live_command() {
|
||||
local name="$1"
|
||||
shift
|
||||
if ! kill -0 "$disk_process" 2>/dev/null; then
|
||||
printf '[disk-observation-unavailable] %s: owned diskutil already exited\n' "$name" >> "$output"
|
||||
elif [ ! -x "$1" ]; then
|
||||
printf '[disk-observation-unavailable] %s: %s is unavailable\n' "$name" "$1" >> "$output"
|
||||
else
|
||||
observe_command "$name" "$@"
|
||||
fi
|
||||
}
|
||||
trap cancel_observation TERM INT
|
||||
printf '[disk-observation] owned-diskutil-child=%s parent-shell=%s\n' "$disk_process" "$$" >> "$output"
|
||||
if [ -x /bin/ps ]; then
|
||||
if kill -0 "$disk_process" 2>/dev/null; then
|
||||
observe_command diskutil-threads /bin/ps -M -p "$disk_process"
|
||||
else
|
||||
printf '[disk-observation-unavailable] diskutil already exited before thread observation\n' >> "$output"
|
||||
fi
|
||||
observe_live_command diskutil-threads-before /bin/ps -M -p "$disk_process"
|
||||
if [ ! -x /usr/bin/sample ]; then
|
||||
printf '[disk-observation-unavailable] diskutil-stack: /usr/bin/sample is unavailable\n' >> "$output"
|
||||
elif ! kill -0 "$disk_process" 2>/dev/null; then
|
||||
printf '[disk-observation-unavailable] diskutil-stack: owned diskutil already exited\n' >> "$output"
|
||||
elif [ -e "$stack_output" ] || [ -L "$stack_output" ]; then
|
||||
printf '[disk-observation-unavailable] diskutil-stack: output already exists\n' >> "$output"
|
||||
elif : > "$stack_output"; then
|
||||
printf '[disk-stack-attempt] owned-diskutil-child=%s duration=1s interval=100ms limit=60s output=%s observation-only=true\n' "$disk_process" "$stack_output" >> "$output"
|
||||
observe_command diskutil-stack /usr/bin/sample "$disk_process" 1 100 -mayDie -file "$stack_output"
|
||||
printf '[disk-stack-result] status=%s observation-only=true; raw report is captured by the Linux host\n' "$OBSERVATION_EXIT" >> "$output"
|
||||
else
|
||||
printf '[disk-observation-unavailable] /bin/ps is unavailable\n' >> "$output"
|
||||
printf '[disk-observation-unavailable] diskutil-stack: output cannot be created\n' >> "$output"
|
||||
fi
|
||||
observe_live_command storagekit-live /bin/launchctl print system/com.apple.storagekitd
|
||||
observe_live_command diskutil-threads-after /bin/ps -M -p "$disk_process"
|
||||
}
|
||||
|
||||
stop_disk_observation() {
|
||||
@@ -226,7 +246,7 @@ run_command() {
|
||||
# Isolate only the failed UID gate; every other watchdog remains unchanged.
|
||||
[[ "$name" != uid ]] || command_limit=180
|
||||
# BEGIN disk IPC diagnostic
|
||||
if [[ "$name" == disks && "${attempt:-0}" == 1 ]]; then command_limit=120; fi
|
||||
if [[ "$name" == disks && "${attempt:-0}" == 1 ]]; then command_limit=600; fi
|
||||
# END disk IPC diagnostic
|
||||
LAST_OUTPUT="/tmp/native-diagnostic-$name.out"
|
||||
printf '\n[proof-command] %s:' "$name" >&3
|
||||
@@ -314,8 +334,9 @@ os_version="$SCALAR"
|
||||
flush_outputs || finish false diagnostic_log_budget_exceeded
|
||||
|
||||
# BEGIN disk IPC diagnostic
|
||||
printf '[disk-diagnostic-runtime] bash=%s stack-observation-only=true\n' "$BASH_VERSION" >&3
|
||||
run_command arbitration_before /bin/launchctl print system/com.apple.diskarbitrationd
|
||||
run_command management_before /bin/launchctl print system/com.apple.diskmanagementd
|
||||
run_command management_before /bin/launchctl print system/com.apple.storagekitd
|
||||
run_command media_before /usr/sbin/ioreg -r -c IOMedia -l -w 0
|
||||
flush_outputs || finish false diagnostic_log_budget_exceeded
|
||||
|
||||
|
||||
Reference in New Issue
Block a user