26 KiB
macOS 13 KVM/Cryptex/NoAVX diagnostic and prepared Full flow
Full RAW run 4219 at d1594e90b3fa9c6f3bb52f12b754ae933105b8c8 verified RAW sector equality, targeted file-cache eviction, KVM, macOS 13.6/x86_64/root and successful cleanup, but all eight disk-list attempts timed out. Installation and application tests were not reached. The next host-only repair retains the completed Recovery hash once after the existing staging marker; it avoids rereading the immutable 711-MB DMG at every poll. Failed or missing-image captures are not retained as success and can retry. Guest code, assets, CPU, memory and deadlines are unchanged. This repair does not yet prove native readiness or CI success.
In readiness mode, this separate manual candidate probes Recovery readiness on the existing Ubuntu Docker daemon with KVM, the real Intel host CPU and macOS 13. It does not install macOS, erase a disk, install .NET or Apple CLT, or run Meeting Assistant. Passing proves only a fresh macOS 13+ x86_64 Recovery guest with root identity, a working launchd system domain, DiskArbitration and exactly one writable 64-GiB guest disk.
Baseline: bootstrap commit 4606de069678e8f95dfe3c7dad1bf5ce5384d30c; separate branch codex/macos-ci-kvm-compatibility. KVM, CPU passthrough, Recovery major version and guest Cryptex staging change together. This is a compatibility experiment, not a causal single-variable A/B test. The TCG/bootstrap experiment remains separate.
The isolated Full NoAVX candidate starts at a356b88ae4a0a26d68098460df86038f9831c080 on codex/macos-native-full-kvm-noavx. Its Compatibility code, archive/staging/configuration rejection checks, host-memory admission and captured-proof heartbeat are transferred from the offline-verified Recovery candidate fef676c810cf78b39aabb872546a76e8ac2b29d5. The additional NoAVX boot kext is the only guest change. Application/tests/specs, Apple bootstrap/installer, payload/SDK pins, owned-disk guards, TRX requirements, CPU/KVM/macOS 13 and guest/container limits are unchanged. Existing phase budgets remain Recovery 40, installation 80, toolchain 30 and tests 25 minutes within the 172-minute host/180-minute job limits. No successful native run is implied by preparation.
The further isolated branch codex/macos-native-full-kvm-noavx-raw starts from that completed Full candidate, 5f686c5c80b6bbb525745de173b57c6393f58bf0. Its optional --recovery-format raw transfers the exact producer, backend selection and existing six producer/three backend fixtures from RAW repair cae38b014f3278a5b939ff980b0138bff5d6b509. The manual Full workflow selects RAW; the controller default remains DMG. Against this baseline, the only additional guest variable is the Recovery disk backend. KVM/host CPU, macOS 13, NoAVX/Cryptex, bootstrap/installer, application/tests and every resource/phase limit are retained. The run/validation comparison metadata names this Full baseline; the unchanged boot-assets receipt continues to describe the original NoAVX component comparison.
In RAW mode the existing QEMU converts the same already-patched DMG, requires sector equality and unchanged DMG SHA256, then retains both images and their hash/equality receipt in owned storage. Before RAM admission, two existing GNU dd calls synchronize and request removal only of those verified files' caches (oflag=nocache conv=nocreat,notrunc,fdatasync count=0); either failure rejects preparation. This avoids the locally reproduced cgroup file-cache admission failure without a package, global cache clearing or larger memory limit. QEMU still attaches the same readonly virtio device and I/O thread with explicit format=raw. Offline Full/source validation and the generated real-QEMU fixture can validate preparation; they establish no guest boot, installation or native test result.
Use dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --run --full --recovery-format raw --output artifacts/native-macos-full. Offline checks use --validate --full --recovery-format raw --source <pristine-pinned-dockur-checkout> --cryptex-archive <verified-Cryptex-ZIP> --noavx-archive <verified-NoAVX-ZIP> --output <fresh-folder>. The generated raw-recovery-real-qemu-fixture.sh accepts an already-patched disposable writable DMG copy and destination; it does not boot a guest. Cleanup remains the existing --cleanup --output artifacts/native-macos-full and removes only the run's owned resources.
The upstream NoAVX AVXpel 12.6 ZIP is pinned to OCLP commit f40057a5292f4804b51bcfe78d5047c7302a6434, 98,356 bytes and locally verified SHA256 b5d6319d0a1f335684a92ecf23369bc3deb776be19e92b0a40860021409d20df. Its two expected bundle files, identity/version and OSBundleRequired=Root are verified. After existing Lilu/Cryptex, Kernel.Add enables NoAVXFSCompressionTypeZlib-AVXpel.kext with Arch=x86_64, executable Contents/MacOS/NoAVXFSCompressionTypeZlib, plist Contents/Info.plist, MinKernel=22.0.0 and empty MaxKernel; both file copies enter the existing SHA256SUMS checks. This is a filesystem-decompression hypothesis, not proof of the current readiness hang's cause.
Memory admission requires the unchanged 4-GiB guest plus 512 MiB QEMU overhead. The copied four fixtures accept run 4204's captured 5,138,696 KiB and reject 4 GiB, missing or invalid availability. This reserves no memory against other host workloads. The existing one-minute heartbeat reads only retained guest-proof.log, printing at most its latest two start/completion/result/version markers, capped at 256 characters each; no new guest query or timer is added. Five existing offline fixtures exercise those bounds. Pushes on this candidate branch skip the PR/Push workflow; pull-request and manual triggers remain.
Reasons and remaining gaps
The existing daemon's Intel Celeron 1037U lacks AVX/AVX2; a separate diagnostic proved KVM enabled/paused state and clean exit. CPU_MODEL=host preserves actual instruction availability rather than advertising AVX2 through emulated Skylake. This candidate refuses a TCG or CPU-model fallback.
All four Swift helpers target x86_64-apple-macos13.0; the macOS 14 EventKit call has an existing macOS 13 fallback. Inspected native Mach-O files in pinned .NET SDK 10.0.401 x64 declare minos 12.0. These source/binary minima are not runtime qualification or vendor support: macOS 13 is outside Microsoft's current .NET 10 supported-OS policy. This probe does not install that SDK, compile helpers or test calendar/audio permissions.
Official CryptexFixup 1.0.5 activates without AVX2 and registers for normal, installer/Recovery and safe-mode boots. It redirects installer/updater ramrod to Apple Silicon's Rosetta Cryptex and bypasses APFS root-hash authentication on Ventura and newer. It does not emulate missing instructions. This kernel patch affects only the owned guest, never a host module.
Recovery cache gap: CryptexFixup does not replace an already running Recovery BaseSystem shared cache. Its installer/update selector targets the installed Cryptex, but this readiness-only run invokes no installer. Staging or loading it therefore proves no Recovery userland compatibility. Actual CPU/kernel behavior, guest injection, all native gates and any later installed-Cryptex/build/test behavior remain unqualified until observed.
Apple Recovery uses the pinned public InternetRecovery protocol with board ID and session/asset tokens, without Apple ID or workstation credentials. The macOS 13 selection, downloaded hash and actual guest version are retained; the hook downloads no full installer or SDK.
Entry point and dependencies
Orchestration/validation remain the .NET 10 file-based app tools/ci/MacOsNativeDiagnostic.cs. Bash/Python stay only in the existing pinned Linux/macOS boot integration.
dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --help
dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --validate
dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --validate --source /path/to/clean/pinned/dockur-clone --cryptex-archive /path/to/CryptexFixup-1.0.5-RELEASE.zip --noavx-archive /path/to/NoAVXFSCompressionTypeZlib-AVXpel-v12.6.zip --output /path/to/fresh/validation
--validate checks result/container contracts without Docker. With --source it verifies the actual Cryptex ZIP/bundle, source seams, generated OpenCore configuration and staging/checksum contracts, checks Bash syntax, then exercises four raw/zlib Recovery fixtures and twelve rejection cases with independent C# CRC32 readback. It also checks preservation of a successful resource snapshot after a later failed capture, leaving the supplied source untouched. It does not download/extract the LongQT ISO, verify a complete Apple Recovery image or execute the active-Lilu runtime checks. The ISO checksum is enforced during the later Docker build; active Lilu and EFI-copy checks execute only during container boot. The optional local Cryptex ZIP must match the release size/hash; omitting it downloads only the public 69,703-byte release. Use a fresh output directory. Dependencies are .NET 10, Git, Bash and Python 3 with its standard library; manual execution also requires the existing Linux/x64 Docker daemon and its existing KVM device.
The optional --noavx-archive supplies the exact local NoAVX ZIP; omitting it downloads the pinned small archive. The unchanged NoAVX validation checks staged bytes and actual generated Kernel.Add, including four invalid archives, two staging failures and five configuration rejections. Use --validate --full with the same arguments to exercise the existing Full bootstrap, installer, disk and TRX contracts as well; MacOsNativeGuest.cs --validate checks the unchanged guest payload/tar/fresh-test-result contracts. These checks do not install an SDK or execute Apple frameworks.
The manual-only workflow keeps these owned run/cleanup entry points; validation invokes neither:
dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --run --output artifacts/native-macos
dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --cleanup --output artifacts/native-macos
Exact bootasset contract
Dockur stays pinned to 16a5b470cdd601bae8b05b02d748d7edfb36c12e. Original Recovery patcher/staging, Dockerfile, OpenCore script and active config hashes are verified before edits. Both existing QEMU image digests remain pinned; other existing upstream downloads are observed through image identity. source-hashes.json includes the generated Recovery patcher, both original/replacement daemon variants and udif_checksums.py, staged from tools/ci/macos-native-udif-checksums.py. This small Python module belongs to the existing Linux UDIF runtime; C# supplies orchestration, validation fixtures and an independent CRC32 implementation.
Run 4173 at 45d7bde71f9f5a1f7121585fe3ee9fc81f7c585f failed before QEMU started: the full macOS 14 plist pattern was absent from the macOS 13 download. The original image's hash was not retained. An independently downloaded comparison for the same board Mac-4B682C642B45593E is macOS 13.6/22G120, Apple product 042-23155, 710,918,897 bytes, SHA256 c19bd12f5cb1651b87b74d04f02a636da762ea46b81c7ebc9f205fa2a976d599. Its Apple chunklist signature and chunks verified before any changes. It is comparison evidence, not the missing run-4173 image identity.
The HFS+ catalog identifies /System/Library/LaunchDaemons/com.apple.recoveryosd.plist as file ID 57231, logical size 465 bytes and one 4,096-byte allocated block. Its exact XML SHA256 is af9d7f6c1948079bd4384d27b6882678d6fb4e338fcf6a8be8f84fceef174ad6. This variant has ProcessType=Interactive; the previous macOS 14 variant has App. The patch accepts only these two exact layouts with exactly one daemon label and original ProgramArguments=[/usr/libexec/recoveryosd]. It preserves each variant's fields and process type, removes only the XML doctype to fit the wrapper arguments, and pads to the original file size. Unknown, duplicate, malformed or wrong-argument layouts fail before image writes. The early rc.cdrom hook remains mount-only; both the unchanged read-only wrapper and guarded Full wrapper exec the original Apple daemon.
The checksum binding validates the original flattened UDIF boundaries and CRC32 values, stages every recompressed chunk before writing, then updates only the changed mish CRC32 and koly data-fork/master CRC32. libdmg-hfsplus provides the checksum semantics; an independent C# reader matched all eight mish checksums on the unchanged comparison. Raw and inflated zlib bytes enter logical CRCs in run order; observed IGNORE runs are omitted. Unobserved ZERO runs, other compression/checksum types, overlaps and invalid boundaries are rejected. Base64 characters are replaced within the same metadata region, preserving its whitespace, length, partition tables and trailer offsets; the entire modified image is read again to verify CRCs. Apple chunklist authentication applies exclusively to the unchanged input, not the deliberately modified guest image. CRC integrity proves no Apple authenticity or native runtime gate.
The original LongQT v0.7 template, 15,884,288 bytes, is now Docker-ADD-checksummed to SHA256 287328995d4198f1b05166f087d85bf7ef66bedafe150d17ad112ac8de60051d. Runtime copies actual EFI_RELEASE/EFI/OC/Kexts, including Lilu 1.7.1, even with official OpenCore DEBUG executables. Active Lilu: executable 526,984 bytes, SHA256 0c016d93cfe40c7fa3965813175c1b991a76f3d295efd5be66ae712b4a3ffb52; Info.plist SHA256 fc885f3319f326e3af60e7965a5216b671772d39d40993ec695758bb43d6ea3a. Staging checks both hashes and bundle version. Cryptex declares Lilu 1.4.7; Lilu history includes Ventura/Sonoma installer/Recovery support before 1.7.1. Existing Lilu is kept.
CryptexFixup-1.0.5-RELEASE.zip, 69,703 bytes, SHA256 25041d94a0fe9a0261caf0ba89b36dfcb21682bf3c697a34bcaddc839576ab30, is checked in C#. Only expected Info.plist/executable files are accepted; identity/version/dependency and individual hashes are recorded. Runtime checks files before/after copying into fresh guest EFI.
Active /assets/config.plist receives exactly one enabled Cryptex immediately after enabled Lilu, preserving every other kext's order. Entry: Arch=x86_64, BundlePath=CryptexFixup.kext, ExecutablePath=Contents/MacOS/CryptexFixup, PlistPath=Contents/Info.plist, MinKernel=22.0.0, empty MaxKernel. OpenCore Kernel.Add requires dependencies first; bounds are Darwin versions. Runtime rechecks order/enabled/paths/architecture/bounds and rejects unverified /custom.plist.
No new force/beta argument is needed for actual no-AVX2 CPUs. Baseline arguments remain. Validation rejects disabling arguments, -crypt_allow_hash_validation (disables the APFS patch) and unexpected Cryptex force/beta overrides. Manifest/profile enter the boot signature; this candidate always rebuilds boot.img and accepts no old cache as evidence.
Gates, privileges and cleanup
The read-only Apple wrapper is byte-identical to baseline: background /Volumes/installstate/readiness.sh then exec /usr/libexec/recoveryosd under the same launchd job/PID. Full mode uses the separate guarded wrapper described below. Source evidence does not prove Apple's executable ran.
Readiness changes only minimum macOS 14 to 13. Validation normalizes that gate to 14 and requires baseline SHA256 4d428f594dac14eff64ed87b172c81ecf85ac91da8c5460cd6ec4b1d310800c3. Architecture, UID, services, disk size/writability/uniqueness, retries, proof bounds, timers and native-wait/cleanup/flush metrics remain identical. Limits stay 45 seconds per native command, 180 seconds for UID, ten minutes disk readiness, 40 minutes host and 45 minutes workflow.
Container profile: KVM=Y, CPU_MODEL=host, VERSION=13, 4-GiB guest, two guest/host CPUs, 6-GiB memory/swap and 512-MiB shared memory. Fresh anonymous /storage holds the 64-GiB disk; evidence reads /storage/13/setup.dmg. Existing resource budget checks remain.
Only device mapping: exactly /dev/kvm:/dev/kvm:rw. Inspection rejects other devices/permissions, added capabilities, device requests/rules, binds, tmpfs overrides, published ports, host networking, privileged mode, wrong limits, unexpected persistent mounts or changed CPU/OS profile. No host modules, infrastructure, secrets, SSH or app lifecycle actions are involved. Guest slirp networking remains.
Evidence retains run/profile identity, source/assets, EFI staging, container/resources, macOS 13 Recovery hash, native proof/result/outcome and cleanup. [recovery-original] logs the exact download's size/SHA256 before modifying it, including when patch failure later deletes the source. guest-container-resources.last-success.stdout.log and its timestamp/hash receipt preserve the last successful resource snapshot independently of a later failed stopped-container docker exec. Optional final Unix HMP capture includes info kvm, info status and a bounded PPM exported from /tmp; capture success passes no native gate.
Both cleanup paths keep exact token/label/ID checks. docker rm --force --volumes removes only the owned container and anonymous volume, then its exact image; no unrelated objects or pruning. Evidence stays seven days. Full native CI still needs a subsequent actual installed remote guest to build/sign helpers and pass the full suite, including five native tests without skips.
Experimental full guest flow
The prepared .gitea/workflows/pr-push-build-and-test.yaml requires macos-native-full after both existing Wine and portable jobs succeed, using ubuntu-latest, a 180-minute limit and the same checkout/SDK/Full-run/always-cleanup/artifact steps as the separate manual .gitea/workflows/macos-native-full.yaml. Both use the existing KVM device, host CPU and macOS 13/Cryptex profile above. Recovery compatibility and the installed build/test path remain unqualified. A Full run must wait for actual remote Recovery qualification, then repeat readiness in its own VM; it cannot accept another run's disk receipt. The full acceptance review follows actual remote build/test verification. The ordinary diagnostic workflow remains read-only.
dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --validate --full --source /path/to/clean/pinned/dockur-clone --cryptex-archive /path/to/CryptexFixup-1.0.5-RELEASE.zip --output /path/to/fresh/full-validation
dotnet run --file tools/ci/MacOsNativeGuest.cs -- --validate
dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --run --full --output artifacts/native-macos-full
dotnet run --file tools/ci/MacOsNativeDiagnostic.cs -- --cleanup --output artifacts/native-macos-full
The host requires a clean exact Git HEAD, creates its Git/PAX source archive and SHA-256, and downloads macOS/x64 SDK 10.0.401 from Microsoft's release URL with the fixed official SHA-512 recorded in both helpers. Source archive, SDK and helper files enter the image and newly owned anonymous /storage volume; no workstation bind mount is introduced. The persistent state is /storage/13/ci-state through the existing guest 9p share. An uncommitted integration cannot be qualified through git archive HEAD: only an archive of the final reviewed commit binds the actual integrated source.
The Full-only macos-native-full-bootstrap.sh waits up to 120 one-second mount attempts for the share's own run.owner; a present foreign or invalid owner fails immediately. Before backgrounding the probe, it exclusively creates and validates the complete one-record probe.started marker bound to run token and source commit. A subsequent launchd wrapper start with the same complete owned marker starts no second child and always execs the original Apple recoveryosd. Foreign, empty, partial, extra-record or unterminated markers and genuine write failures remain errors; markers are preserved. An initial child failure remains a token-bound bootstrap failure, without silent retry. If the share never appears, the wrapper reports a bounded mount failure to stderr without writing to foreign state, then still execs Apple. This contract relies on completing the marker before the wrapper's first Apple exec; it does not prove arbitrary simultaneous installer starts or actual guest 9p atomicity.
Before the only guest eraseDisk, C# revalidates the owned Docker boundary, sole anonymous storage mount, exact writable 64-GiB raw image, live QEMU attachment and per-run emulated disk serial. Only after a valid fresh native Recovery receipt does it atomically provide the run/commit/disk permit. The guarded Apple installer rechecks diskutil and the corresponding IORegistry serial; missing or ambiguous identity fails. Its exclusive persistent started marker binds token, commit and disk. A duplicate child with a complete matching marker exits neutrally, preserving the active installation phase. Foreign or invalid guards and real write errors publish installation failure. The installer child never starts an extra Apple daemon; it terminates while the Full wrapper preserves the original one. The local losing-claim fixture publishes a complete record between checks; the low-level create-before-printf window is not claimed to be a general concurrent-race solution.
With mounted run-owned state, fail(), nonzero startosinstall and TERM/INT atomically publish a token-bound installation-failed phase for the next host poll, preserving the erase guard. The host observes Full bootstrap failure even before the install permit. Upstream startosinstall, USR1 bootstrap staging, Setup Assistant/admin packages and byte-for-byte staging checks remain in use. Installer reboots preserve the same QEMU process, disk, NVRAM and share. The read-only Recovery media stays attached. There is no automatic container restart or erase retry.
The existing firstboot LaunchDaemon invokes macos-native-firstboot.sh before staging cleanup. This Bash seam is required because the guest has Apple boot tools but no .NET SDK yet. It proves installed APFS / maps through one APFS container and physical store to the same owned 64-GiB whole disk, mounts the state share, installs a compatible Apple CLT catalog label through headless softwareupdate, and verifies the CLT package/compiler. It records the actual xcrun SDK version/path and compiles and runs a macOS 13-targeted smoke program importing AppKit, AVFoundation, ScreenCaptureKit, EventKit and WebKit. CLT compatibility is established by these actual compiler/framework gates, rather than a guessed catalog version. There is no GUI fallback, Apple account or new secret. macos-native-disk-guard.sh holds the shared pre-.NET Apple disk/IORegistry check. The pinned upstream Python UDIF patcher remains the image-format runtime binding; exact patch matches and compressed-slot checks fail closed.
After verifying and extracting the SDK on the guest's own APFS work directory, MacOsNativeGuest.cs validates payload hashes, safe Git tar paths and PAX commit, then restores/builds/tests net10.0 with TZ=Europe/Berlin. It records the actual SDK/compiler environment and requires installed macOS 13+ x86_64 with guest root identity. It requires fresh outputs for all four Swift helpers, actual Mach-O/x86_64 tools output and strict audio-app codesign verification. Only fresh TRX with 577 total/executed/passed results, zero failures/skips and all five named macOS tests explicitly passed is accepted. TRX SHA-256 uses the same raw-byte snapshot as parsing, including any UTF-8 BOM. Binary minima and local parser fixtures do not demonstrate .NET vendor support or installed runtime compatibility.
The Full outer deadline is 172 minutes; the workflow declares 180 minutes within the existing three-hour server limit, leaving time for evidence and owned-resource cleanup. Independent budgets are Recovery 40 minutes, installer 80, firstboot/CLT 30 and guest checks/restore/build/tests 25; the outer deadline also bounds their combined runtime and preparation. The same 4-GiB/two-CPU guest and 6-GiB container remain, with ALLOCATE=N. Full execution checks 32 GiB of existing Docker free space before downloads/boot and 8 GiB of guest free space before toolchain work. Insufficient resources, networking, Apple catalog availability, disk ownership, installer progress or test proof fail without changing infrastructure.
Artifacts include source/SDK hashes, pinned boot patches, installer/Apple/firstboot logs, CLT SDK identity, installed-root/disk identity with APFS mapping plists, native tool logs, guest phase and Full-result receipts, helper hashes and binary-preserved TRX. Polling prints a bounded heartbeat each elapsed minute with phase/time/budget, liveness and readiness. Cleanup retains exact saved resource ID/ownership checks through finally and workflow always(); only the owned container/image/anonymous volume are removed. Installed files disappear with that volume; retained CI evidence remains outside it. Shared Docker build cache is not pruned.
Local --validate --full generates source/fixture evidence and executes the generated installer guard and complete Full wrapper with harmless filesystem/mount, child-process and Apple-exec boundary fixtures. It covers owned/foreign/invalid/write-failed installer and probe markers, restart, first-child failure and delayed/missing/foreign-owner shares, plus all four positive/twelve negative Recovery image cases and retained resource-snapshot checks. Its independent C# CRC32 reader validates fixture readback while the existing pinned Python UDIF binding performs the patch. It starts no Docker or VM, erases no disk, installs no OS/toolchain, builds no application and runs no native test. Bash syntax, synthetic disk parser and receipt tests do not qualify actual Apple exec, guest storage or native CI. Read-only --compression-chunk evidence applies only to its retained Recovery chunk, not the new Full wrapper or a different downloaded macOS image. This remains a locally prepared candidate until actual remote guests satisfy every native gate.