{"benchmarks":[{"id":"f186a392-d655-481f-894f-b09c081dba85","setterAccountId":"0c745d95-e708-43a8-9f5e-0348f3f18774","challengeId":null,"trackName":null,"status":"open","name":"layr-labs/ssi-ordering-challenge","description":"Fill-reducing elimination ordering for sparse symmetric indefinite matrices; scored as a size-bucketed weighted geomean of predicted factorization flops vs feral AMD (lower is better).","category":"optimization","direction":"-","editablePaths":["src/ordering"],"setupCommand":["bash","-lc","command -v cargo >/dev/null 2>&1 || { echo 'Rust toolchain (cargo) not found. Install from https://rustup.rs' >&2; exit 1; }; installed_cargo_deny=\"$(cargo deny --version 2>/dev/null || true)\"; if [ \"$installed_cargo_deny\" != \"cargo-deny 0.20.2\" ]; then cargo install cargo-deny --version 0.20.2 --locked --force; fi; [ \"$(cargo deny --version 2>/dev/null)\" = \"cargo-deny 0.20.2\" ] || { echo 'cargo-deny 0.20.2 is required after setup' >&2; exit 1; }; bash scripts/prepare-build.sh && cargo build --release -p ssi-ordering-challenge"],"benchmarkCommand":["bash","-lc","bash scripts/local-candidate-build.sh && cargo run --release"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":10,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/Layr-Labs/ssi-ordering-challenge","sourceBranch":"main","sourceRef":"0cb0501b46869870334499b35d0f918e5187a301","rootDir":null,"baselineScore":0.834283,"baselineMetrics":{"geomean_fill_ratio":0.940316,"geomean_flop_ratio":0.834283},"currentBestScore":0.784202,"currentBestMetrics":{"geomean_fill_ratio":0.915023,"geomean_flop_ratio":0.784202},"setupSnapshotRef":"gha:a077d42c5e505a36bd8ee31b9755cfb1fc1e2266","closesAt":null,"openedAt":"2026-08-27T18:39:13.922Z","createdAt":"2026-08-27T18:29:25.903Z","updatedAt":"2026-08-29T14:55:52.843Z","deletedAt":null},{"id":"be15cf17-1dc9-4d1a-9b24-264e23575de9","setterAccountId":"1d517260-7249-445d-8e9e-a7122ac00966","challengeId":null,"trackName":null,"status":"open","name":"davidtai/mlxfast-gemma4-26b-a4b","description":"Gemma 4 26B A4B MLX text-tower inference speedup track (mlx-community/gemma-4-26B-A4B-it-qat-4bit; sparse MoE, 128 routed experts, 8 per token; 5 full-attention layers of 30 on a six-layer repeat; tied embeddings). Optimize the MLX runner, the offline transform, and the vendored MLX/Metal kernel families the forward pass dispatches. Also optimize the stateless assistant speculative-decode arm: a 4-layer Q-only drafter that borrows the target's embeddings, hidden state and KV. A second speculative arm runs a DFlash drafter staged at dflash-head/, single-stream. You set the draft depth in code. Each speculative arm reads one editable constant submissionDraftDepth. Both constants default to 1. benchd adds no default. MTP: CBv2MTPRoundDriver.submissionDraftDepth is a ceiling on the adaptive depth controller. The controller selects a depth from 0 to this ceiling each round. The MTP envelope limits the ceiling to 3 (maxDraftTokens 3; maxAutomaticRectangularTokens 32 at batch 8). DFlash: DFlashDraftModel.submissionDraftDepth is a fixed block depth. Block diffusion drafts one whole block each round. The ceiling is the recommended block size of the drafter minus one. The engine limits it to 15. On DFlash the acceptance varies, not the drafted depth. A value above the ceiling of an arm clamps to the ceiling. The engine does not refuse it. The depth that actually ran is sealed per run as effective_spec, alongside the realized draft length effective_mean_draft_len. The DFlash arm is scored: the track fixture's allowed_modes is serial, mtp and dflash, and a submission runs whichever mode its own code drives. A submission selects the arm with the 'arm' key of spec-decoder-head.manifest.json ('dflash' or 'mtp'; an absent key and an absent file both mean 'mtp', and any other value is refused by name before any measurement). The lever is therefore to make the drafter and target forward passes and the surrounding kernel dispatch faster. Both speculative heads are the organizer's pinned weights. A participant may re-quantize either head. A re-quantization happens on load, in memory: each head loader already calls quantize(model:) while it binds the checkpoint, and both files that hold that call are editable paths (Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Gemma4MTP.swift for the MTP assistant, Vendor/mlx-swift-lm/Libraries/MLXSpeculative/DFlashDraftModel.swift for the DFlash drafter). Nothing on disk changes; docs/participant-contract.md section 4.4 states the mechanism. Custom head weights are not accepted on this track: mtp-head/ and dflash-head/ are not editable paths, so a submission carries no head weight file, and a head declaration accepts source 'pinned' only. The two declaration files, mtp-head.manifest.json and spec-decoder-head.manifest.json, stay editable and stay optional. An absent declaration selects the organizer-pinned head. The target model may never be re-quantized. The KV backend is pinned contiguous on both legs and refuses rather than degrades. Scoring for the serial and MTP arms is a composite over a batched B=8 cohort, composite = prefill_gain^0.25 * decode_gain^0.75. The DFlash arm is single-stream only -- the cohort driver refuses it by name -- so a DFlash submission is measured and scored in the single-stream series instead, over the same pinned pool, and the two arms are never pooled into one value. docs/participant-contract.md section 5 is the authority.","category":"swift","direction":"+","editablePaths":["mtp-head.manifest.json","spec-decoder-head.manifest.json","Sources/MLXFastModel","Sources/MLXFastTransform","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Gemma4.swift","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Gemma4Text.swift","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Gemma4MTP.swift","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Gemma4MTPTarget.swift","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Gemma4MTPConfigurationValidation.swift","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Gemma4CBv2MTPDrafter.swift","Vendor/mlx-swift-lm/Libraries/MLXSpeculative/DFlashDraftModel.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/ContinuousBatchingV2","Vendor/mlx-swift-lm/Libraries/MLXLMServer/Runtime/ToolStreamHandler.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/AttentionUtils.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BaseConfiguration.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableRotatingKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/DynamicSlice.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/Evaluate.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/JSONDecodingTypes.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/KVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/LanguageModel.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEApplication.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEUtils.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/SwitchLayers.swift","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/copy.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_axis.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_front.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemv.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/softmax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/sort.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/arg_reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_axis.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_front.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/indexing.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rms_norm.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rope.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/scaled_dot_product_attention.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sdpa_vector.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/attn","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/gemm","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary_ops.h"],"setupCommand":["bash","-c","./tools/fetch-benchd.sh && ./setup.sh"],"benchmarkCommand":["bash","-c","MLXFAST_SCORE_PATH=score.json ./tools/gemma4-measure-and-score.sh"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/Layr-Labs/mlxfast-gemma4-26b-a4b-engine","sourceBranch":"main","sourceRef":"288488940097fb5219a21b5f778303650418363f","rootDir":null,"baselineScore":0.949382852205473,"baselineMetrics":{"series":"batched_free_run_v1_2_b8","composite":{"decode_gain":0.9188215838092533,"prefill_gain":1.0473020827061343,"composite_score":0.9493828522054727,"composite_speedup_floor":0.9,"composite_speedup_floor_met":true},"parity_ok":true,"cohort_sha256":"aac0d031edfac22077f0b62bef9348599edb5d0f2a8039481a39002d2e08d748","effective_spec":{"mtp":{"depth":3},"mode":"mtp"},"scored_batch_size":8,"decode_token_total":1024,"raw_ratio_of_means":0.9512618901175841,"accepted_pair_count":4,"prefill_token_total":8192,"effective_mean_draft_len":1.0078740157480315,"non_drafting_round_count":126,"serial_seconds_per_token_mean":0.006211692494140625,"candidate_seconds_per_token_mean":0.006529949910400391,"serial_decode_window_seconds_mean":4.5925807705,"serial_prefill_window_seconds_mean":1.7681923435,"candidate_decode_window_seconds_mean":4.998337927,"candidate_prefill_window_seconds_mean":1.68833078125},"currentBestScore":1.88802660882816,"currentBestMetrics":{"series":"batched_free_run_v1_2_b8","composite":{"decode_gain":2.05468806600179,"prefill_gain":1.4648553698621225,"composite_score":1.8880266088281614,"composite_speedup_floor":0.9,"composite_speedup_floor_met":true},"parity_ok":true,"cohort_sha256":"aac0d031edfac22077f0b62bef9348599edb5d0f2a8039481a39002d2e08d748","effective_spec":{"mtp":{"depth":3},"mode":"mtp"},"scored_batch_size":8,"decode_token_total":1024,"raw_ratio_of_means":1.8465066700579666,"accepted_pair_count":4,"prefill_token_total":8192,"effective_mean_draft_len":1,"non_drafting_round_count":128,"serial_seconds_per_token_mean":0.006222877990478515,"candidate_seconds_per_token_mean":0.003370081511962891,"serial_decode_window_seconds_mean":4.58800796875,"serial_prefill_window_seconds_mean":1.7842190935,"candidate_decode_window_seconds_mean":2.2329462290000004,"candidate_prefill_window_seconds_mean":1.21801723925},"setupSnapshotRef":"gha:647c3cf03df77ab59776240eba00f14fc79b3132","closesAt":null,"openedAt":"2026-08-27T14:24:15.189Z","createdAt":"2026-08-27T14:11:24.811Z","updatedAt":"2026-08-29T14:22:22.369Z","deletedAt":null},{"id":"a2e3eaa8-95c0-4a62-81d3-2cd7e78e8575","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":"c62905f6-c846-48b1-8eae-24390ea195e4","trackName":"irs-reduction-threshold-lower","status":"open","name":"proximity-prize/proximity-prize/irs-reduction-threshold-lower","description":"Maximise the kernel-checked induced spot-check-bit floor at a certified-safe IRS reduction radius.","category":"lean","direction":"+","editablePaths":["ProximityPrize/SubmissionLower"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh lower"],"scorePath":".yukon/irs-reduction-threshold-lower-score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark-irs-reduction-threshold-lower.yml"},"maxConcurrentWorkflows":2,"minScoreImprovementBips":null,"maxSubmissionBytes":4194304,"sourceUrl":"https://github.com/proximity-prize/proximity-prize","sourceBranch":"main","sourceRef":"baf2c0c2a54fbb4072e4c7006c09660b546def26","rootDir":null,"baselineScore":53,"baselineMetrics":{"unit":"bits","track":"irs-reduction-threshold-lower","commit":"a36cdb6a0c9be507b70276a92129bb15d4ed2a27","metric":"reduction_lower_centibits","radius":0.25,"verified":true,"centibits":5300,"challenge":"proximity-prize-reduction-lower","radiusExact":"1/4","proofMetrics":{"radius_numerator":1,"radius_denominator":4},"submissionId":"b16f439c-89b5-4d3b-8f28-2a7efef1d201","launchEligible":true,"challengeVersion":"irs-reduction-threshold-v1","independentVerified":true,"locallyKernelChecked":true,"verificationAuthority":"independent-verifier"},"currentBestScore":67.32,"currentBestMetrics":{"unit":"bits","track":"irs-reduction-threshold-lower","commit":"baf2c0c2a54fbb4072e4c7006c09660b546def26","metric":"reduction_lower_centibits","radius":0.3054923713207245,"verified":true,"centibits":6732,"challenge":"proximity-prize-reduction-lower","radiusExact":"10250623/33554432","proofMetrics":{"radius_numerator":10250623,"radius_denominator":33554432},"submissionId":"f753fbf3-e344-4d1f-98a1-fb579992a847","launchEligible":true,"challengeVersion":"irs-reduction-threshold-v4","independentVerified":true,"locallyKernelChecked":true,"verificationAuthority":"independent-verifier"},"setupSnapshotRef":"gha:a36cdb6a0c9be507b70276a92129bb15d4ed2a27","closesAt":"2026-09-19T15:36:53.689Z","openedAt":"2026-08-20T15:36:53.427Z","createdAt":"2026-08-20T15:32:42.909Z","updatedAt":"2026-08-29T04:42:28.515Z","deletedAt":null},{"id":"1226aa4f-1258-4eb7-8c9e-846ece1e2543","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":"c62905f6-c846-48b1-8eae-24390ea195e4","trackName":"irs-reduction-threshold-upper","status":"open","name":"proximity-prize/proximity-prize/irs-reduction-threshold-upper","description":"Minimise the kernel-checked induced spot-check-bit ceiling at a winning-set-unsafe IRS radius.","category":"lean","direction":"-","editablePaths":["ProximityPrize/SubmissionUpper"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh upper"],"scorePath":".yukon/irs-reduction-threshold-upper-score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark-irs-reduction-threshold-upper.yml"},"maxConcurrentWorkflows":2,"minScoreImprovementBips":null,"maxSubmissionBytes":4194304,"sourceUrl":"https://github.com/proximity-prize/proximity-prize","sourceBranch":"main","sourceRef":"baf2c0c2a54fbb4072e4c7006c09660b546def26","rootDir":null,"baselineScore":128,"baselineMetrics":{"unit":"bits","track":"irs-reduction-threshold-upper","commit":"a36cdb6a0c9be507b70276a92129bb15d4ed2a27","metric":"reduction_upper_centibits","verified":true,"centibits":12800,"challenge":"proximity-prize-reduction-upper","unsafeIndex":131072,"proofMetrics":{"unsafe_index":131072},"submissionId":"f8b277ce-9802-4c9e-a4b6-4676464a7a72","unsafeRadius":0.5,"launchEligible":true,"challengeVersion":"irs-reduction-threshold-v1","unsafeRadiusExact":"131072/262144","independentVerified":true,"locallyKernelChecked":true,"verificationAuthority":"independent-verifier"},"currentBestScore":116.13,"currentBestMetrics":{"unit":"bits","track":"irs-reduction-threshold-upper","commit":"224323f053d0d1dfa78d7b9625194fe0133f04bc","metric":"reduction_upper_centibits","verified":true,"centibits":11613,"challenge":"proximity-prize-reduction-upper","unsafeIndex":122369,"proofMetrics":{"unsafe_index":122369},"submissionId":"1d29c2e8-18dd-4957-9001-fca30db27d6e","unsafeRadius":0.4668006896972656,"launchEligible":true,"challengeVersion":"irs-reduction-threshold-v1","unsafeRadiusExact":"122369/262144","independentVerified":true,"locallyKernelChecked":true,"verificationAuthority":"independent-verifier"},"setupSnapshotRef":"gha:a36cdb6a0c9be507b70276a92129bb15d4ed2a27","closesAt":"2026-09-19T15:36:53.689Z","openedAt":"2026-08-20T15:36:53.427Z","createdAt":"2026-08-20T15:32:42.909Z","updatedAt":"2026-08-29T04:42:28.388Z","deletedAt":null},{"id":"5d1ee4d7-80bd-4555-b182-6505f26ef495","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"open","name":"eigenlabs/qwen38-challenge","description":"Qwen 3.8 27B native-MTP speculative-decode track: make the checkpoint's own multi-token-prediction head decode faster without changing a single emitted token. The competitive surface is a UNION of two things. First, and as in every track of this family, the MLX runner and kernel path: the Qwen 3.8 text tower, the offline transform, and the vendored MLX/Metal kernel families the forward pass dispatches. Second, the WHOLE speculative apparatus: the drafting code (Qwen36MTPBlockSession and its siblings), the draft depth and per-round schedule -- adaptive policies and non-drafting rounds are legal, bounded only by a trusted maximum of 8 drafts per round -- and the MTP HEAD WEIGHTS themselves, declared through mtp-head.manifest.json and fetched or shipped in-branch under mtp-head/. What stays fixed is the organizer-pinned TARGET checkpoint and its transform contract, the tokenizer, the goldens, and the verify/accept/ledger machinery. That split is not a compromise, it is the safety argument: anything that only PROPOSES tokens is safe by construction, because the trusted parent re-checks every emitted token against the hidden serial trajectory and closes the row ledger over the drafts actually proposed. Decode-only paired scoring against the TRUE serial control (MTP depth 0) on the same box, in the same session: the published score is the median of the eight per-prompt raw serial-relative speedups, so serial itself is exactly 1.0 (the ~0.935 an unmodified tree scored was measured on QWEN 3.6 and is not a 3.8 expectation). Native MTP decode is measured exact-greedy against serial on this checkpoint (12/12 runs to 512 tokens, all EOS branches), so the fidelity gate requires exact token match rather than a near-tie budget.","category":"swift","direction":"+","editablePaths":["mtp-head.manifest.json","mtp-head","Sources/MLXFastModel","Sources/MLXFastTransform","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Qwen35.swift","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Qwen35MTP.swift","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Qwen35MoE.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/AttentionUtils.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BaseConfiguration.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BatchKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableRotatingKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompiledDecode.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/DynamicSlice.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/Evaluate.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/JSONDecodingTypes.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/KVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/LanguageModel.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEApplication.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEUtils.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/SwitchLayers.swift","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/copy.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_axis.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_front.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemv.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/softmax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/sort.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/arg_reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_axis.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_front.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/indexing.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rms_norm.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rope.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/scaled_dot_product_attention.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sdpa_vector.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/attn","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/gemm","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary_ops.h"],"setupCommand":["bash","-c","./setup.sh && ./setup-qwen-mtp.sh"],"benchmarkCommand":["bash","-c","MLXFAST_SCORE_PATH=score.json ./benchmark-qwen-mtp.sh --local-iterate"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"qwen-mtp-ranked-benchmark.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/Layr-Labs/qwen-3.8-mtp-challenge","sourceBranch":"main","sourceRef":"0863b06ac16e26e48fc06e97444095b00feb66d4","rootDir":null,"baselineScore":0.992701635158627,"baselineMetrics":{"mode":"qwen-mtp-paired-decode-only","commit":"5d029178765cf727e7ee530b0b4c731d566f908a","mtp_depth":8,"per_prompt":[{"parity_ok":true,"prompt_sha256":"919318e117fd04fd827e4cbc82abc30f37f6aea0e1c2609b27e3679c64c25fca","raw_ratio_of_means":0.9837019082,"accepted_pair_count":1,"head_provenance_sha256":"157f750eb467b5ad24c4bc353f2b959b865d3d0f85b0bd61c72cec4c70235d61","effective_mean_draft_len":1,"non_drafting_round_count":0,"mtp_seconds_per_token_mean":0.0385672266,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379386544},{"parity_ok":true,"prompt_sha256":"192fb6218ae7c1950dffb24b13ce3608857162bcd17853d3892f1d70876ad313","raw_ratio_of_means":0.8471662401,"accepted_pair_count":1,"head_provenance_sha256":"157f750eb467b5ad24c4bc353f2b959b865d3d0f85b0bd61c72cec4c70235d61","effective_mean_draft_len":1,"non_drafting_round_count":0,"mtp_seconds_per_token_mean":0.044782164,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379379375},{"parity_ok":true,"prompt_sha256":"4b9e88cdbe5aa2e76c03dcd76c5d2166e1e89c40ec2f777720aa87b225d7e1ff","raw_ratio_of_means":0.9573773557,"accepted_pair_count":1,"head_provenance_sha256":"157f750eb467b5ad24c4bc353f2b959b865d3d0f85b0bd61c72cec4c70235d61","effective_mean_draft_len":1,"non_drafting_round_count":0,"mtp_seconds_per_token_mean":0.0396350233,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379456738},{"parity_ok":true,"prompt_sha256":"a2ea8b60458057ae731c11ff841eb95ad66f6f785b9bc8a6ef169d59ccca679b","raw_ratio_of_means":1.0017013621,"accepted_pair_count":1,"head_provenance_sha256":"157f750eb467b5ad24c4bc353f2b959b865d3d0f85b0bd61c72cec4c70235d61","effective_mean_draft_len":1,"non_drafting_round_count":0,"mtp_seconds_per_token_mean":0.0378142307,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0378785664},{"parity_ok":true,"prompt_sha256":"00142a4454b01e746b517c35eb23d1f248cd934bc3a743250b03745b9e711986","raw_ratio_of_means":1.0760531533,"accepted_pair_count":1,"head_provenance_sha256":"157f750eb467b5ad24c4bc353f2b959b865d3d0f85b0bd61c72cec4c70235d61","effective_mean_draft_len":1,"non_drafting_round_count":0,"mtp_seconds_per_token_mean":0.0353235702,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0380100391},{"parity_ok":true,"prompt_sha256":"c1ec58669d032878b7fd82d811132bca19e79c91e19bf1414fe47c9b6d16e09e","raw_ratio_of_means":0.9710550323,"accepted_pair_count":1,"head_provenance_sha256":"157f750eb467b5ad24c4bc353f2b959b865d3d0f85b0bd61c72cec4c70235d61","effective_mean_draft_len":1,"non_drafting_round_count":0,"mtp_seconds_per_token_mean":0.0391117383,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379796503},{"parity_ok":true,"prompt_sha256":"ea82dcb5931a2d78a42cebd4fb4189bf875a31c947b67792782e455f88a155a1","raw_ratio_of_means":1.0100745104,"accepted_pair_count":1,"head_provenance_sha256":"157f750eb467b5ad24c4bc353f2b959b865d3d0f85b0bd61c72cec4c70235d61","effective_mean_draft_len":1,"non_drafting_round_count":0,"mtp_seconds_per_token_mean":0.0375415664,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379197793},{"parity_ok":true,"prompt_sha256":"3b10cb4dcae4b7f8222b1594861c4bbcc72aa890d72b46e97caee994884c82a5","raw_ratio_of_means":1.058210699,"accepted_pair_count":1,"head_provenance_sha256":"157f750eb467b5ad24c4bc353f2b959b865d3d0f85b0bd61c72cec4c70235d61","effective_mean_draft_len":1,"non_drafting_round_count":0,"mtp_seconds_per_token_mean":0.0358586211,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379459765}],"aggregation":"median_of_per_prompt_raw_serial_relative_speedup","median_rule":"even_n_mean_of_two_central_order_statistics","prompt_count":8,"score_anchor":"serial = 1.0","decode_tokens":512,"parity_all_ok":true,"pairs_per_prompt":1,"scoring_normalized":false,"accepted_pair_count":8,"mtp_max_draft_depth":8,"decode_speedup_floor":0.9,"qwen_mtp_weights_hash":"b53e4991737cdf50827e518e7559628874d3ff6d5f63bebc057ddbb16a89e2cd","decode_speedup_ceiling":3,"mtp_decode_speedup_min":0.8471662402,"mtp_decode_speedup_median":0.9837019084,"mtp_decode_speedup_raw_median":0.9927016351586265,"candidate_mtp_seconds_per_token_mean":0.03857926759519614,"baseline_serial_seconds_per_token_mean":0.03794453467708081,"mtp_decode_speedup_pooled_ratio_of_means":0.9835473051283542},"currentBestScore":3.7291100105909,"currentBestMetrics":{"mode":"qwen-mtp-paired-decode-only","commit":"0863b06ac16e26e48fc06e97444095b00feb66d4","mtp_depth":8,"per_prompt":[{"parity_ok":true,"prompt_sha256":"919318e117fd04fd827e4cbc82abc30f37f6aea0e1c2609b27e3679c64c25fca","raw_ratio_of_means":3.5487164001,"accepted_pair_count":1,"head_provenance_sha256":"559b24ebca354018e4402fdb1f5af1afe5a0721bd2ebf04133500d846f7d5f71","effective_mean_draft_len":4.381818181818182,"non_drafting_round_count":0,"prefill_seconds_per_token":0.0010290956124663353,"mtp_seconds_per_token_mean":0.0106696133,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0378634317},{"parity_ok":true,"prompt_sha256":"192fb6218ae7c1950dffb24b13ce3608857162bcd17853d3892f1d70876ad313","raw_ratio_of_means":3.9424893496,"accepted_pair_count":1,"head_provenance_sha256":"559b24ebca354018e4402fdb1f5af1afe5a0721bd2ebf04133500d846f7d5f71","effective_mean_draft_len":6.148148148148148,"non_drafting_round_count":0,"prefill_seconds_per_token":0.0010280255228281021,"mtp_seconds_per_token_mean":0.0096309709,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379700002},{"parity_ok":true,"prompt_sha256":"4b9e88cdbe5aa2e76c03dcd76c5d2166e1e89c40ec2f777720aa87b225d7e1ff","raw_ratio_of_means":2.1286204425,"accepted_pair_count":1,"head_provenance_sha256":"559b24ebca354018e4402fdb1f5af1afe5a0721bd2ebf04133500d846f7d5f71","effective_mean_draft_len":2.2976190476190474,"non_drafting_round_count":0,"prefill_seconds_per_token":0.001026480458676815,"mtp_seconds_per_token_mean":0.0177830332,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.037853328},{"parity_ok":true,"prompt_sha256":"a2ea8b60458057ae731c11ff841eb95ad66f6f785b9bc8a6ef169d59ccca679b","raw_ratio_of_means":3.9206429412,"accepted_pair_count":1,"head_provenance_sha256":"559b24ebca354018e4402fdb1f5af1afe5a0721bd2ebf04133500d846f7d5f71","effective_mean_draft_len":5.086956521739131,"non_drafting_round_count":0,"prefill_seconds_per_token":0.0010269258636981249,"mtp_seconds_per_token_mean":0.0098065643,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0384480371},{"parity_ok":true,"prompt_sha256":"00142a4454b01e746b517c35eb23d1f248cd934bc3a743250b03745b9e711986","raw_ratio_of_means":3.9095036211,"accepted_pair_count":1,"head_provenance_sha256":"559b24ebca354018e4402fdb1f5af1afe5a0721bd2ebf04133500d846f7d5f71","effective_mean_draft_len":5.2555555555555555,"non_drafting_round_count":0,"prefill_seconds_per_token":0.0010284902527928352,"mtp_seconds_per_token_mean":0.009695336,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379039512},{"parity_ok":true,"prompt_sha256":"c1ec58669d032878b7fd82d811132bca19e79c91e19bf1414fe47c9b6d16e09e","raw_ratio_of_means":1.2596858762,"accepted_pair_count":1,"head_provenance_sha256":"559b24ebca354018e4402fdb1f5af1afe5a0721bd2ebf04133500d846f7d5f71","effective_mean_draft_len":0.1557377049180328,"non_drafting_round_count":449,"prefill_seconds_per_token":0.0010273202788084745,"mtp_seconds_per_token_mean":0.0301173888,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0379384493},{"parity_ok":true,"prompt_sha256":"ea82dcb5931a2d78a42cebd4fb4189bf875a31c947b67792782e455f88a155a1","raw_ratio_of_means":3.9097165339,"accepted_pair_count":1,"head_provenance_sha256":"559b24ebca354018e4402fdb1f5af1afe5a0721bd2ebf04133500d846f7d5f71","effective_mean_draft_len":4.989247311827957,"non_drafting_round_count":0,"prefill_seconds_per_token":0.0010275174863636494,"mtp_seconds_per_token_mean":0.0096808379,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.037849332},{"parity_ok":true,"prompt_sha256":"3b10cb4dcae4b7f8222b1594861c4bbcc72aa890d72b46e97caee994884c82a5","raw_ratio_of_means":2.4338402226,"accepted_pair_count":1,"head_provenance_sha256":"559b24ebca354018e4402fdb1f5af1afe5a0721bd2ebf04133500d846f7d5f71","effective_mean_draft_len":2.647887323943662,"non_drafting_round_count":0,"prefill_seconds_per_token":0.0010280273854732513,"mtp_seconds_per_token_mean":0.0155653048,"noop_reference_decode_speedup":1,"serial_seconds_per_token_mean":0.0378834649}],"aggregation":"median_of_per_prompt_raw_serial_relative_speedup","median_rule":"even_n_mean_of_two_central_order_statistics","prompt_count":8,"score_anchor":"serial = 1.0","decode_tokens":512,"parity_all_ok":true,"pairs_per_prompt":1,"scoring_normalized":false,"accepted_pair_count":8,"mtp_max_draft_depth":8,"decode_speedup_floor":0.9,"qwen_mtp_weights_hash":"b53e4991737cdf50827e518e7559628874d3ff6d5f63bebc057ddbb16a89e2cd","decode_speedup_ceiling":5,"mtp_decode_speedup_min":1.2596858756,"mtp_decode_speedup_median":3.5487164173,"prefill_seconds_per_token":0.0010277353576384485,"mtp_decode_speedup_raw_median":3.729110010590895,"candidate_mtp_seconds_per_token_mean":0.014118631137534976,"baseline_serial_seconds_per_token_mean":0.03796374931698665,"mtp_decode_speedup_pooled_ratio_of_means":2.688911477831475},"setupSnapshotRef":"gha:5d029178765cf727e7ee530b0b4c731d566f908a","closesAt":"2026-08-30T00:00:00.000Z","openedAt":"2026-08-14T22:36:59.138Z","createdAt":"2026-08-14T21:57:09.582Z","updatedAt":"2026-08-23T11:43:09.658Z","deletedAt":null},{"id":"8abde9cf-9256-4453-99e2-9de8632a7943","setterAccountId":"016e262b-5d20-4f95-a1ed-e208574dc681","challengeId":"fc968480-5224-4183-a46b-3818e86ed515","trackName":"x86","status":"open","name":"eigenlabs/flock-challenge-multi/x86","description":"Optimize Flock's Rust prover sources for maximum BLAKE3 compression-proof throughput on x86_64 CPUs.","category":"rust","direction":"+","editablePaths":["crates/flock-core/src","crates/flock-prover/src"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark-blake3-x86.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":100,"maxSubmissionBytes":8388608,"sourceUrl":"https://github.com/Layr-Labs/flock-challenge-multi","sourceBranch":"main","sourceRef":"5677856a8d303db19e7d297e9c848dbcc0dde637","rootDir":null,"baselineScore":400569.198403517,"baselineMetrics":{"threads":16,"verified":true,"batch_size":262144,"p10_seconds":0.6517771929999999,"proof_bytes":439375,"warmup_runs":20,"measured_runs":100,"trial_seconds":[0.655319051,0.656376185,0.65358972,0.653988998,0.654378744,0.658551612,0.65123356,0.65215646,0.658484131,0.655850859,0.65578767,0.65517627,0.653248098,0.648644274,0.652496607,0.652287715,0.654427194,0.653476116,0.654804653,0.656325513,0.654469448,0.65938532,0.652937041,0.650608441,0.65553343,0.655748127,0.653534343,0.659712178,0.654545178,0.654417742,0.652614422,0.661067103,0.650984251,0.658776974,0.653756048,0.6526212,0.655959174,0.651716263,0.658310641,0.65512428,0.657154054,0.654004318,0.653072741,0.65463476,0.655698267,0.653832277,0.651843377,0.65331981,0.657785849,0.65148166,0.657164182,0.65433794,0.654938624,0.656241866,0.655326295,0.654318082,0.65137808,0.648633023,0.655228621,0.657152602,0.652785033,0.649454718,0.655801266,0.656169831,0.653096117,0.65495834,0.652236247,0.657692239,0.655409551,0.653483103,0.653854079,0.652551969,0.65386735,0.65795177,0.652625928,0.656697771,0.652022533,0.652620006,0.6542619,0.654645571,0.656680662,0.657506214,0.657611019,0.653833717,0.65751725,0.653165149,0.654430307,0.656486304,0.654014272,0.653120847,0.653603627,0.65131458,0.658959947,0.652726391,0.657108067,0.654833712,0.660889159,0.654642269,0.656683524,0.651783963],"median_seconds":0.6544287505,"warmup_trial_seconds":[0.65663691,0.657763292,0.659617152,0.652309833,0.651958647,0.654728674,0.653920904,0.654079628,0.652375808,0.659703236,0.653641988,0.65346963,0.655930944,0.65230921,0.653462829,0.652105807,0.657153741,0.653749917,0.655164787,0.660385722],"p90_p10_latency_ratio":1.009244337121198,"aggregate_compressions_per_second":400433.4051024325},"currentBestScore":1534641.41672299,"currentBestMetrics":{"threads":16,"verified":true,"batch_size":262144,"p10_seconds":0.1698660036,"proof_bytes":439503,"warmup_runs":20,"measured_runs":100,"trial_seconds":[0.170302016,0.171617612,0.173429307,0.169869548,0.169989894,0.170740405,0.170781913,0.171316161,0.171915565,0.17098477,0.170178654,0.169842506,0.169193582,0.169741539,0.171449458,0.170544712,0.169897204,0.173088367,0.171030825,0.170102416,0.169866858,0.169407015,0.169570994,0.170315003,0.172105078,0.169634512,0.171225842,0.171663371,0.170113444,0.172531384,0.170279379,0.17021297,0.170945634,0.171500775,0.170170784,0.171708733,0.173945091,0.170592828,0.171231256,0.17076113,0.171694377,0.170583471,0.170742938,0.171943468,0.172235116,0.17172303,0.170797783,0.17112389,0.169847207,0.172044687,0.170929796,0.170190879,0.172791091,0.169995116,0.170051659,0.170625331,0.170681655,0.171400719,0.169924239,0.171202183,0.170925728,0.171075735,0.170231786,0.173931487,0.17049086,0.171678185,0.170946086,0.172530745,0.170822286,0.170985027,0.170059408,0.171252819,0.170942129,0.170883545,0.171214835,0.170005806,0.17086449,0.169991899,0.171029537,0.170991386,0.17060462,0.170813222,0.171111255,0.169858314,0.172383606,0.1701222,0.169290687,0.171248477,0.17014329,0.170801593,0.170277485,0.170028424,0.169553444,0.170240435,0.171943373,0.172943055,0.173183748,0.170878508,0.170843783,0.170301109],"median_seconds":0.170817754,"warmup_trial_seconds":[0.171212265,0.170611607,0.170767288,0.170497589,0.169848658,0.17253047,0.171087563,0.17098968,0.170003179,0.170732061,0.170386287,0.171693235,0.171372472,0.172697567,0.171147899,0.17145756,0.170100139,0.17042489,0.171771143,0.170035077],"p90_p10_latency_ratio":1.0140343644371226,"aggregate_compressions_per_second":1533745.6715247452},"setupSnapshotRef":"gha:d25765bf1b47827e62490aedb331bf66e8b684ed","closesAt":null,"openedAt":"2026-08-14T14:44:21.482Z","createdAt":"2026-08-14T14:34:05.548Z","updatedAt":"2026-08-28T23:34:03.797Z","deletedAt":null},{"id":"c6931fd9-f783-4e1f-84df-c7ffe49c3426","setterAccountId":"016e262b-5d20-4f95-a1ed-e208574dc681","challengeId":null,"trackName":null,"status":"closed","name":"eigenlabs/lighter-prover-challenge","description":"Optimize verified private active-witness throughput; public synthetic scores are provisional smoke results only.","category":"rust","direction":"+","editablePaths":["Cargo.toml","Cargo.lock","circuit","bench","vendor"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":67108864,"sourceUrl":"https://github.com/Layr-Labs/lighter-prover-challenge","sourceBranch":"master","sourceRef":"ce919deaf8c8820b7d066feabb2ea2a2aec5dcb0","rootDir":null,"baselineScore":3.18419941491498,"baselineMetrics":{"runtime":"official-throughput","cacheable":false,"fixture_id":"ranked-v1","competitive":true,"provisional":false,"transactions":2500,"candidate_sha":"fbdc64c39be2b8668d67b30ee0a24c342a133f06","fixture_count":5,"benchmark_case":"private-active-ranked","expected_proofs":5,"proving_seconds":785.126706666,"verified_proofs":5,"verifier_sha256":"3418587865c584eb153ae84e77ba197b1f93b3cf4710f95b7364e020a3c3a5f5","protocol_version":"lighter-mixed-block-proof-v1","timing_authority":"trusted verifier parent","verifier_source_rev":"381fd529eb61dfff9ad245d94fce214a0a64d927","transactions_per_second":3.184199414914977,"transaction_count_source":"parsed-active-witness"},"currentBestScore":33.6546578925448,"currentBestMetrics":{"runtime":"official-throughput","cacheable":false,"fixture_id":"ranked-v1","competitive":true,"provisional":false,"transactions":2500,"candidate_sha":"ce919deaf8c8820b7d066feabb2ea2a2aec5dcb0","fixture_count":5,"benchmark_case":"private-active-ranked","expected_proofs":5,"proving_seconds":74.28392254,"verified_proofs":5,"verifier_sha256":"3418587865c584eb153ae84e77ba197b1f93b3cf4710f95b7364e020a3c3a5f5","protocol_version":"lighter-mixed-block-proof-v1","timing_authority":"trusted verifier parent","verifier_source_rev":"381fd529eb61dfff9ad245d94fce214a0a64d927","transactions_per_second":33.65465789254483,"transaction_count_source":"parsed-active-witness"},"setupSnapshotRef":"gha:fbdc64c39be2b8668d67b30ee0a24c342a133f06","closesAt":"2026-08-25T23:00:00.000Z","openedAt":"2026-08-05T14:28:58.397Z","createdAt":"2026-08-03T23:40:17.289Z","updatedAt":"2026-08-25T23:00:01.134Z","deletedAt":null},{"id":"8806afb8-8dfa-4a9d-8f11-4735c5fde7c9","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"open","name":"eigenlabs/flock-challenge","description":"Optimize Flock's Rust prover sources for maximum BLAKE3 compression-proof throughput.","category":"rust","direction":"+","editablePaths":["crates/flock-core/src","crates/flock-prover/src"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark-blake3-mac.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":8388608,"sourceUrl":"https://github.com/Layr-Labs/flock-challenge","sourceBranch":"main","sourceRef":"c35c1c474650cd72ccbdc7280b0f4bb3ee1ab0dd","rootDir":null,"baselineScore":516419.997403834,"baselineMetrics":{"threads":10,"verified":true,"batch_size":262144,"p10_seconds":0.5009957710999999,"proof_bytes":439311,"warmup_runs":20,"measured_runs":100,"trial_seconds":[0.497840041,0.501168042,0.495852834,0.499477166,0.501237625,0.504038958,0.499011625,0.499883625,0.501012333,0.500681208,0.506251583,0.50287775,0.504539709,0.502551583,0.50459725,0.503786583,0.505605875,0.501206917,0.503737667,0.506604667,0.504335125,0.499921375,0.50199,0.501858583,0.502236125,0.50575775,0.504717292,0.502451416,0.502957416,0.505718875,0.506581167,0.503100375,0.500999417,0.500510583,0.503630792,0.500962958,0.505910042,0.5018095,0.507082958,0.515119375,0.506395667,0.500110792,0.505617583,0.50393425,0.502533333,0.507111542,0.507481916,0.508595333,0.508796709,0.506638584,0.504828,0.510224583,0.509746541,0.507086541,0.508411583,0.510165042,0.505593209,0.509411209,0.512189208,0.5128895,0.50775375,0.513486708,0.512062,0.509981083,0.508178875,0.513229167,0.51019,0.511408042,0.51218375,0.512930709,0.511856125,0.523485458,0.51125525,0.511421167,0.512809541,0.513042542,0.514005834,0.524294625,0.51452625,0.515516625,0.514486083,0.517153917,0.51774225,0.51535525,0.517679,0.514982458,0.519363583,0.5154365,0.514622,0.516241959,0.519153667,0.515319333,0.520671083,0.51855125,0.517326958,0.517951334,0.516799792,0.523073333,0.518937125,0.520928417],"median_seconds":0.5076178330000001,"warmup_trial_seconds":[0.498059333,0.499914041,0.505435167,0.497384375,0.503595834,0.501038333,0.498874542,0.499102666,0.500434791,0.50327625,0.500989333,0.501598625,0.498612458,0.50561375,0.497712167,0.499630333,0.500158584,0.500654042,0.502242708,0.5002665],"p90_p10_latency_ratio":1.0334681214238297,"aggregate_compressions_per_second":515192.09719956765},"currentBestScore":1851759.05313334,"currentBestMetrics":{"threads":10,"verified":true,"batch_size":262144,"p10_seconds":0.1406087712,"proof_bytes":439535,"warmup_runs":20,"measured_runs":100,"trial_seconds":[0.140660792,0.143233125,0.144012542,0.141541167,0.139870584,0.140955875,0.141406792,0.141245,0.1413435,0.142788792,0.143301667,0.141465958,0.140005417,0.14311,0.140554458,0.141944834,0.142943125,0.141099667,0.141499292,0.141578208,0.14155925,0.144848833,0.141570458,0.141267208,0.141379166,0.1402745,0.143247792,0.141531625,0.1407205,0.144076417,0.141306958,0.142104875,0.141274,0.141025625,0.141538875,0.142209625,0.142597,0.141169083,0.141308292,0.140607459,0.142791333,0.142054708,0.14369325,0.141879542,0.14226675,0.143171417,0.141241042,0.14110725,0.142299625,0.141519375,0.141407167,0.141913084,0.140153834,0.1410865,0.142261542,0.142172667,0.140608917,0.142880666,0.140803625,0.140711875,0.142005792,0.140531458,0.141295709,0.14191975,0.140590917,0.141208292,0.145747833,0.1405375,0.142755042,0.141194208,0.141825959,0.143990833,0.140400833,0.142142083,0.14149125,0.143675125,0.142709042,0.141109333,0.141864833,0.141371584,0.143028708,0.142068208,0.141242125,0.141410708,0.142569083,0.142808708,0.142535,0.1429455,0.143124125,0.145822084,0.142875875,0.143023,0.141260834,0.141220667,0.141146875,0.141660375,0.14217725,0.140980416,0.14329925,0.142321166],"median_seconds":0.14156485400000002,"warmup_trial_seconds":[0.145626917,0.141552292,0.14255925,0.140789708,0.142995958,0.142443375,0.141498875,0.141003125,0.141149542,0.141658792,0.141456375,0.1418265,0.141908542,0.141848292,0.1429915,0.140746916,0.143367834,0.144003166,0.142298083,0.141201417],"p90_p10_latency_ratio":1.018805132691466,"aggregate_compressions_per_second":1847113.4483527364},"setupSnapshotRef":"gha:d79e043ea2d7757237f9e9bf9d5e6d57d26cc73e","closesAt":null,"openedAt":"2026-07-30T00:59:48.945Z","createdAt":"2026-07-30T00:52:05.481Z","updatedAt":"2026-08-27T22:49:25.948Z","deletedAt":null},{"id":"1854efdf-feba-4773-bae9-b80520881a74","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"closed","name":"eigenlabs/mlxfast-challenge","description":"Optimize Poolside Laguna XS 2.1 NVFP4 MoE inference by transforming weights offline and reducing compute latency without failing the correctness gate.","category":"swift","direction":"+","editablePaths":["Sources/MLXFastModel","Sources/MLXFastTransform","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Laguna.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/KVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/LanguageModel.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/Evaluate.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEUtils.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEApplication.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableRotatingKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompiledDecode.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/DynamicSlice.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BatchKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BaseConfiguration.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/JSONDecodingTypes.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/SwitchLayers.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/AttentionUtils.swift","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/matmul.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/jit_kernels.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/scaled_dot_product_attention.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sdpa_vector.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/arg_reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/atomic.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_col.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_init.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_row.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_all.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/indexing.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_front.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_axis.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/gemm","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/attn","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp4.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp8.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rope.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rms_norm.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.h","Vendor/mlx-swift/Source/Cmlx/mlx-generated/softmax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/copy.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/sort.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_front.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_axis.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemv.cpp"],"setupCommand":["bash","-c","./setup.sh"],"benchmarkCommand":["bash","-c","MLXFAST_SCORE_PATH=score.json ./benchmark.sh --local-iterate"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/Layr-Labs/mlxfast-challenge","sourceBranch":"main","sourceRef":"4ea72c3b28873fca23b12b6f33193a2eeb5042f8","rootDir":null,"baselineScore":1.00340857549622,"baselineMetrics":{"error":"","commit":"38fcfff9c865ff45244cb491629108d913e35d77","runtime":"swift","timestamp":"2026-07-24T01:08:21Z","case_count":7,"num_layers":40,"golden_hash":"95d59b28c0d83f4702e9861c2f6c80d5fca649281e1e0d7010895a8e13e5dfff","peak_ram_gb":21,"actual_token":null,"harness_hash":"8a187c29626fe347984663c0fca3cd3818ea6a86b1672c9a750814a3190b555f","max_abs_diff":0,"weights_hash":"aff994300573c5e8589563fc9ff57cdcfb1ef9b49e14898be290a75a6b294b3d","checked_steps":512,"decode_speedup":1.0197579721957408,"expected_token":null,"partial_result":false,"expert_hit_rate":0,"prefill_speedup":0.955916385082656,"bandwidth_source":"ram_resident_model","gpqa_ttft_passed":true,"gpqa_ttft_source":"hidden_gpqa_first_token","expert_bytes_read":0,"expert_cache_hits":0,"gpqa_ttft_seconds":0.43,"preflight_seconds":0.00016,"first_failing_case":null,"first_failing_step":null,"passed_correctness":true,"weights_byte_count":21568891382,"weights_file_count":9,"correctness_seconds":22,"expert_cache_misses":0,"expert_read_seconds":0,"first_failing_layer":null,"semantic_gpqa_model":"claude-opus-4-8","decode_speedup_floor":0.95,"gpqa_ttft_case_count":5,"gpqa_ttft_pass_count":5,"semantic_gpqa_passed":true,"gpqa_ttft_max_seconds":1.3,"gpqa_ttft_p50_seconds":0.16,"prefill_speedup_floor":0.95,"bandwidth_gb_per_token":0,"benchmark_wall_seconds":44,"expert_cache_evictions":0,"timed_benchmark_seconds":25,"decode_seconds_per_token":0.013560619140625,"semantic_gpqa_case_count":5,"semantic_gpqa_pass_count":2,"prefill_seconds_per_token":0.000381130126953125,"expert_peak_cached_tensors":0,"process_resident_memory_gb":0.092,"passed_decode_speedup_floor":true,"passed_prefill_speedup_floor":true,"baseline_decode_seconds_per_token":0.0138285494765625,"baseline_prefill_seconds_per_token":0.000364328533203125},"currentBestScore":2.6195531094824,"currentBestMetrics":{"error":"","commit":"4ea72c3b28873fca23b12b6f33193a2eeb5042f8","runtime":"swift","timestamp":"2026-08-11T09:21:33Z","case_count":11,"num_layers":40,"golden_hash":"be7738fccd6a28807ae7d18c038cbbc9e1b05dab26b99b2f247358fdc67fcf71","peak_ram_gb":21,"actual_token":null,"harness_hash":"20df518660750c2f28572faf34691fe48abdd96fe1cbe1eeb5e2d6550e03d754","max_abs_diff":0,"weights_hash":"aff994300573c5e8589563fc9ff57cdcfb1ef9b49e14898be290a75a6b294b3d","checked_steps":1344,"decode_speedup":2.823562202783501,"expected_token":null,"partial_result":false,"expert_hit_rate":0,"prefill_speedup":2.0917835943913303,"bandwidth_source":"ram_resident_model","gpqa_ttft_passed":true,"gpqa_ttft_source":"hidden_gpqa_first_token","expert_bytes_read":0,"expert_cache_hits":0,"gpqa_ttft_seconds":0.14,"preflight_seconds":0.00014,"first_failing_case":null,"first_failing_step":null,"passed_correctness":true,"weights_byte_count":21568891382,"weights_file_count":9,"correctness_seconds":40,"expert_cache_misses":0,"expert_read_seconds":0,"first_failing_layer":null,"semantic_gpqa_model":"claude-opus-4-8","decode_speedup_floor":0.95,"gpqa_ttft_case_count":9,"gpqa_ttft_pass_count":9,"semantic_gpqa_passed":true,"gpqa_ttft_max_seconds":0.47,"gpqa_ttft_p50_seconds":0.073,"prefill_speedup_floor":0.95,"bandwidth_gb_per_token":0,"benchmark_wall_seconds":59,"expert_cache_evictions":0,"timed_benchmark_seconds":52,"decode_seconds_per_token":0.0049034762421875,"semantic_gpqa_case_count":9,"semantic_gpqa_pass_count":9,"prefill_seconds_per_token":0.000188171712890625,"expert_peak_cached_tensors":0,"process_resident_memory_gb":0.11,"passed_decode_speedup_floor":true,"passed_prefill_speedup_floor":true,"baseline_decode_seconds_per_token":0.0138452701796875,"baseline_prefill_seconds_per_token":0.000393614501953125},"setupSnapshotRef":"gha:38fcfff9c865ff45244cb491629108d913e35d77","closesAt":"2026-08-11T17:00:00.000Z","openedAt":"2026-07-24T01:13:38.399Z","createdAt":"2026-07-24T00:45:03.305Z","updatedAt":"2026-08-12T01:06:26.062Z","deletedAt":null},{"id":"a89b9cfe-1625-423b-bf38-f369c8cd5492","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"failed","name":"gpsanant/mlxfast-challenge-dev-serial-v2-2","description":"Optimize Poolside Laguna XS 2.1 NVFP4 MoE inference by transforming weights offline and reducing compute latency without failing the correctness gate.","category":"swift","direction":"+","editablePaths":["Sources/MLXFastModel","Sources/MLXFastTransform","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Laguna.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/KVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/LanguageModel.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/Evaluate.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEUtils.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEApplication.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableRotatingKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompiledDecode.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/DynamicSlice.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BatchKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BaseConfiguration.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/JSONDecodingTypes.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/SwitchLayers.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/AttentionUtils.swift","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/matmul.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/jit_kernels.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/scaled_dot_product_attention.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sdpa_vector.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/arg_reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/atomic.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_col.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_init.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_row.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_all.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/indexing.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_front.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_axis.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/gemm","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/attn","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp4.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp8.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rope.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rms_norm.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.h","Vendor/mlx-swift/Source/Cmlx/mlx-generated/softmax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/copy.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/sort.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_front.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_axis.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemv.cpp"],"setupCommand":["bash","-c","./setup.sh"],"benchmarkCommand":["bash","-c","MLXFAST_SCORE_PATH=score.json ./benchmark.sh --local-iterate"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/Layr-Labs/mlxfast-challenge","sourceBranch":null,"sourceRef":"461af5b76fc0b4f037150134ad4c424df095c846","rootDir":null,"baselineScore":null,"baselineMetrics":null,"currentBestScore":null,"currentBestMetrics":null,"setupSnapshotRef":null,"closesAt":null,"openedAt":null,"createdAt":"2026-07-23T21:40:37.435Z","updatedAt":"2026-07-23T22:41:12.165Z","deletedAt":null},{"id":"d5ad0a15-9e0f-46d3-bd0e-9cedd2111b2b","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"failed","name":"gpsanant/mlxfast-challenge-dev-serial-v2","description":"Optimize Poolside Laguna XS 2.1 NVFP4 MoE inference by transforming weights offline and reducing compute latency without failing the correctness gate.","category":"swift","direction":"+","editablePaths":["Sources/MLXFastModel","Sources/MLXFastTransform","Vendor/mlx-swift-lm/Libraries/MLXLLM/Models/Laguna.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/KVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/LanguageModel.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/Evaluate.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEUtils.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/RoPEApplication.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompilableRotatingKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/CompiledDecode.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/DynamicSlice.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BatchKVCache.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/BaseConfiguration.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/JSONDecodingTypes.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/SwitchLayers.swift","Vendor/mlx-swift-lm/Libraries/MLXLMCommon/AttentionUtils.swift","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/matmul.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/jit_kernels.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/scaled_dot_product_attention.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sdpa_vector.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/softmax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/copy.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/unary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/binary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/ternary_ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/arg_reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/sort.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduce_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/atomic.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_col.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_init.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_row.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/ops.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/reduction/reduce_all.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/indexing.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_front.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/indexing/gather_axis.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/gemm","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/steel/attn","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/quantized_utils.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp4.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp8.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.h","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/fp_quantized_nax.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rope.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/rms_norm.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.metal","Vendor/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/kernels/gemv.h","Vendor/mlx-swift/Source/Cmlx/mlx-generated/softmax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/copy.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/unary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/binary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/ternary_ops.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/sort.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/reduce_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_front.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gather_axis.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/quantized_utils.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/fp_quantized_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemm_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_fused_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_splitk_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_gemm_gather_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/steel_attention_nax.cpp","Vendor/mlx-swift/Source/Cmlx/mlx-generated/gemv.cpp"],"setupCommand":["bash","-c","./setup.sh"],"benchmarkCommand":["bash","-c","MLXFAST_SCORE_PATH=score.json ./benchmark.sh --local-iterate"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/Layr-Labs/mlxfast-challenge","sourceBranch":null,"sourceRef":"461af5b76fc0b4f037150134ad4c424df095c846","rootDir":null,"baselineScore":null,"baselineMetrics":null,"currentBestScore":null,"currentBestMetrics":null,"setupSnapshotRef":null,"closesAt":null,"openedAt":null,"createdAt":"2026-07-23T21:40:16.765Z","updatedAt":"2026-07-23T21:40:20.470Z","deletedAt":null},{"id":"6180ffe0-2cda-4a34-ae4c-37a8708a661e","setterAccountId":"c6b0320e-ce02-41b3-87f3-10381b59276c","challengeId":null,"trackName":null,"status":"closed","name":"openfrontiercs/pack-the-polyominoes","description":"Pack a set of 100–10,000 polyominoes (each 1–10 cells) into an axis-aligned rectangle using rotations, reflections, and translations with no overlaps, minimizing the rectangle area A = W × H. Scored 0–1 by area ratio against the reference packing across 70 hidden cases. Time limit 2s, memory 256 MB.","category":"algorithmic","direction":"+","editablePaths":["solution.cpp"],"setupCommand":["bash","-lc",".yukon/setup.sh"],"benchmarkCommand":["bash","-lc",".yukon/run.sh"],"scorePath":".yukon/score.json","networkPolicy":null,"runner":{"provider":"frontiercs"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":65536,"sourceUrl":"https://github.com/eigenlabs-research/frontiercs-benchmarks","sourceBranch":null,"sourceRef":"447c1c0f723c7c8db6cd635ccae1008ced09fe0b","rootDir":"algorithmic/problems/0","baselineScore":0,"baselineMetrics":{"cases":[{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":425163,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":421415,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":416564,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":524612,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":305999,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":193592,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":597178,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":458178,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":589301,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":638706,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":337272,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":237199,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":613511,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":390245,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":1652714,"memory":344064,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":498765,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":1143303,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":189335,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":449576,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":419839,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":572212,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":544144,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":367083,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":221007,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":464167,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":449946,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":646775,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":765698,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":1156961,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":2199474,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":791298,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":1992923,"memory":528384,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":608507,"memory":401408,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":433003,"memory":266240,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":748212,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":1805154,"memory":405504,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":559614,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":646902,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":276582,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":427427,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":471173,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":533708,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":586127,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":311798,"memory":299008,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":567366,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":501152,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":214062,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":876474,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":231152,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":391879,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":593542,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":407285,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":387671,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":148222,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":645259,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":465762,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":404403,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":495369,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":293772,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":198619,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":190119,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":189411,"memory":294912,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":242104,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":302601,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":616308,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":166359,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int64 expected\n","time":382843,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0}],"score":0,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":0},"currentBestScore":0.984680809477612,"currentBestMetrics":{"cases":[{"ok":false,"msg":"points 0.9988014981 Ratio: 0.998801498 (cells=6667, W=15, H=445, area=6675)\n","time":40599688,"memory":2899968,"status":"Wrong Answer","scoreRatio":0.998801498,"scoreRatioUnbounded":0.998801498},{"ok":false,"msg":"points 0.9599525043 Ratio: 0.959952504 (cells=8893, W=48, H=193, area=9264)\n","time":1782360374,"memory":4964352,"status":"Wrong Answer","scoreRatio":0.959952504,"scoreRatioUnbounded":0.959952504},{"ok":false,"msg":"points 0.9991516437 Ratio: 0.999151644 (cells=4711, W=23, H=205, area=4715)\n","time":21049457,"memory":2510848,"status":"Wrong Answer","scoreRatio":0.999151644,"scoreRatioUnbounded":0.999151644},{"ok":false,"msg":"points 0.968613382 Ratio: 0.968613382 (cells=49408, W=49, H=1041, area=51009)\n","time":1730362959,"memory":18939904,"status":"Wrong Answer","scoreRatio":0.968613382,"scoreRatioUnbounded":0.968613382},{"ok":false,"msg":"points 0.9642626877 Ratio: 0.964262688 (cells=10469, W=47, H=231, area=10857)\n","time":1804569761,"memory":5181440,"status":"Wrong Answer","scoreRatio":0.964262688,"scoreRatioUnbounded":0.964262688},{"ok":false,"msg":"points 0.997260274 Ratio: 0.997260274 (cells=2184, W=15, H=146, area=2190)\n","time":7682101,"memory":1851392,"status":"Wrong Answer","scoreRatio":0.997260274,"scoreRatioUnbounded":0.997260274},{"ok":false,"msg":"points 0.9663214809 Ratio: 0.966321481 (cells=21089, W=44, H=496, area=21824)\n","time":1821425141,"memory":8921088,"status":"Wrong Answer","scoreRatio":0.966321481,"scoreRatioUnbounded":0.966321481},{"ok":false,"msg":"points 0.9994219653 Ratio: 0.999421965 (cells=5187, W=10, H=519, area=5190)\n","time":31214365,"memory":2600960,"status":"Wrong Answer","scoreRatio":0.999421965,"scoreRatioUnbounded":0.999421965},{"ok":false,"msg":"points 0.9995598592 Ratio: 0.999559859 (cells=4542, W=16, H=284, area=4544)\n","time":21981024,"memory":2473984,"status":"Wrong Answer","scoreRatio":0.999559859,"scoreRatioUnbounded":0.999559859},{"ok":false,"msg":"points 0.9688945132 Ratio: 0.968894513 (cells=62017, W=36, H=1778, area=64008)\n","time":1779909516,"memory":23101440,"status":"Wrong Answer","scoreRatio":0.968894513,"scoreRatioUnbounded":0.968894513},{"ok":false,"msg":"points 0.962454469 Ratio: 0.962454469 (cells=10305, W=43, H=249, area=10707)\n","time":1798032942,"memory":5107712,"status":"Wrong Answer","scoreRatio":0.962454469,"scoreRatioUnbounded":0.962454469},{"ok":false,"msg":"points 0.9961165049 Ratio: 0.996116505 (cells=1539, W=15, H=103, area=1545)\n","time":5216051,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.996116505,"scoreRatioUnbounded":0.996116505},{"ok":false,"msg":"points 0.9521690768 Ratio: 0.952169077 (cells=5136, W=31, H=174, area=5394)\n","time":1770524598,"memory":3268608,"status":"Wrong Answer","scoreRatio":0.952169077,"scoreRatioUnbounded":0.952169077},{"ok":false,"msg":"points 0.9615635889 Ratio: 0.961563589 (cells=8831, W=41, H=224, area=9184)\n","time":1777830337,"memory":4730880,"status":"Wrong Answer","scoreRatio":0.961563589,"scoreRatioUnbounded":0.961563589},{"ok":false,"msg":"points 0.9606815203 Ratio: 0.960681520 (cells=5864, W=28, H=218, area=6104)\n","time":1772103292,"memory":3710976,"status":"Wrong Answer","scoreRatio":0.96068152,"scoreRatioUnbounded":0.96068152},{"ok":false,"msg":"points 0.9885964912 Ratio: 0.988596491 (cells=1127, W=19, H=60, area=1140)\n","time":4011533,"memory":1810432,"status":"Wrong Answer","scoreRatio":0.988596491,"scoreRatioUnbounded":0.988596491},{"ok":false,"msg":"points 0.9950310559 Ratio: 0.995031056 (cells=1602, W=14, H=115, area=1610)\n","time":5487892,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.995031056,"scoreRatioUnbounded":0.995031056},{"ok":false,"msg":"points 0.9906291834 Ratio: 0.990629183 (cells=1480, W=18, H=83, area=1494)\n","time":5140394,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.990629183,"scoreRatioUnbounded":0.990629183},{"ok":false,"msg":"points 0.9971198157 Ratio: 0.997119816 (cells=1731, W=14, H=124, area=1736)\n","time":5804212,"memory":1851392,"status":"Wrong Answer","scoreRatio":0.997119816,"scoreRatioUnbounded":0.997119816},{"ok":false,"msg":"points 0.9932508436 Ratio: 0.993250844 (cells=1766, W=14, H=127, area=1778)\n","time":5090891,"memory":1949696,"status":"Wrong Answer","scoreRatio":0.993250844,"scoreRatioUnbounded":0.993250844},{"ok":false,"msg":"points 0.9965517241 Ratio: 0.996551724 (cells=1734, W=15, H=116, area=1740)\n","time":5499734,"memory":1949696,"status":"Wrong Answer","scoreRatio":0.996551724,"scoreRatioUnbounded":0.996551724},{"ok":false,"msg":"points 0.9932785661 Ratio: 0.993278566 (cells=1330, W=13, H=103, area=1339)\n","time":4587484,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.993278566,"scoreRatioUnbounded":0.993278566},{"ok":false,"msg":"points 0.9959954233 Ratio: 0.995995423 (cells=1741, W=19, H=92, area=1748)\n","time":5689249,"memory":1949696,"status":"Wrong Answer","scoreRatio":0.995995423,"scoreRatioUnbounded":0.995995423},{"ok":false,"msg":"points 0.9886178862 Ratio: 0.988617886 (cells=1216, W=15, H=82, area=1230)\n","time":4268988,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.988617886,"scoreRatioUnbounded":0.988617886},{"ok":false,"msg":"points 0.9964497041 Ratio: 0.996449704 (cells=1684, W=13, H=130, area=1690)\n","time":6011617,"memory":1945600,"status":"Wrong Answer","scoreRatio":0.996449704,"scoreRatioUnbounded":0.996449704},{"ok":false,"msg":"points 0.9931773879 Ratio: 0.993177388 (cells=1019, W=18, H=57, area=1026)\n","time":3521296,"memory":1556480,"status":"Wrong Answer","scoreRatio":0.993177388,"scoreRatioUnbounded":0.993177388},{"ok":false,"msg":"points 0.9752791069 Ratio: 0.975279107 (cells=1223, W=19, H=66, area=1254)\n","time":4425042,"memory":1703936,"status":"Wrong Answer","scoreRatio":0.975279107,"scoreRatioUnbounded":0.975279107},{"ok":false,"msg":"points 0.9894736842 Ratio: 0.989473684 (cells=1504, W=19, H=80, area=1520)\n","time":5227586,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.989473684,"scoreRatioUnbounded":0.989473684},{"ok":false,"msg":"points 0.9952830189 Ratio: 0.995283019 (cells=1899, W=18, H=106, area=1908)\n","time":6786535,"memory":1982464,"status":"Wrong Answer","scoreRatio":0.995283019,"scoreRatioUnbounded":0.995283019},{"ok":false,"msg":"points 0.9846889952 Ratio: 0.984688995 (cells=1029, W=19, H=55, area=1045)\n","time":3727436,"memory":1560576,"status":"Wrong Answer","scoreRatio":0.984688995,"scoreRatioUnbounded":0.984688995},{"ok":false,"msg":"points 0.994057725 Ratio: 0.994057725 (cells=1171, W=19, H=62, area=1178)\n","time":3967081,"memory":1839104,"status":"Wrong Answer","scoreRatio":0.994057725,"scoreRatioUnbounded":0.994057725},{"ok":false,"msg":"points 0.9961215255 Ratio: 0.996121526 (cells=1541, W=13, H=119, area=1547)\n","time":7378587,"memory":1851392,"status":"Wrong Answer","scoreRatio":0.996121526,"scoreRatioUnbounded":0.996121526},{"ok":false,"msg":"points 0.9940828402 Ratio: 0.994082840 (cells=1512, W=13, H=117, area=1521)\n","time":4960776,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.99408284,"scoreRatioUnbounded":0.99408284},{"ok":false,"msg":"points 0.993442623 Ratio: 0.993442623 (cells=1515, W=25, H=61, area=1525)\n","time":208217,"memory":1822720,"status":"Wrong Answer","scoreRatio":0.993442623,"scoreRatioUnbounded":0.993442623},{"ok":false,"msg":"points 0.9982269504 Ratio: 0.998226950 (cells=1689, W=12, H=141, area=1692)\n","time":5438194,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.99822695,"scoreRatioUnbounded":0.99822695},{"ok":false,"msg":"points 0.9952120383 Ratio: 0.995212038 (cells=1455, W=17, H=86, area=1462)\n","time":4916052,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.995212038,"scoreRatioUnbounded":0.995212038},{"ok":false,"msg":"points 0.9870660313 Ratio: 0.987066031 (cells=1450, W=13, H=113, area=1469)\n","time":4648835,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.987066031,"scoreRatioUnbounded":0.987066031},{"ok":false,"msg":"points 0.9852744311 Ratio: 0.985274431 (cells=1472, W=18, H=83, area=1494)\n","time":5084965,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.985274431,"scoreRatioUnbounded":0.985274431},{"ok":false,"msg":"points 0.9928229665 Ratio: 0.992822967 (cells=1660, W=22, H=76, area=1672)\n","time":5725292,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.992822967,"scoreRatioUnbounded":0.992822967},{"ok":false,"msg":"points 0.974927676 Ratio: 0.974927676 (cells=1011, W=17, H=61, area=1037)\n","time":3604537,"memory":1556480,"status":"Wrong Answer","scoreRatio":0.974927676,"scoreRatioUnbounded":0.974927676},{"ok":false,"msg":"points 0.9511378849 Ratio: 0.951137885 (cells=1421, W=18, H=83, area=1494)\n","time":13555071,"memory":2342912,"status":"Wrong Answer","scoreRatio":0.951137885,"scoreRatioUnbounded":0.951137885},{"ok":false,"msg":"points 0.9966978536 Ratio: 0.996697854 (cells=1811, W=23, H=79, area=1817)\n","time":6075494,"memory":1961984,"status":"Wrong Answer","scoreRatio":0.996697854,"scoreRatioUnbounded":0.996697854},{"ok":false,"msg":"points 0.9910714286 Ratio: 0.991071429 (cells=1221, W=14, H=88, area=1232)\n","time":4076434,"memory":1810432,"status":"Wrong Answer","scoreRatio":0.991071429,"scoreRatioUnbounded":0.991071429},{"ok":false,"msg":"points 0.9935897436 Ratio: 0.993589744 (cells=1550, W=12, H=130, area=1560)\n","time":5154454,"memory":1904640,"status":"Wrong Answer","scoreRatio":0.993589744,"scoreRatioUnbounded":0.993589744},{"ok":false,"msg":"points 0.9506666667 Ratio: 0.950666667 (cells=1426, W=25, H=60, area=1500)\n","time":21849587,"memory":2342912,"status":"Wrong Answer","scoreRatio":0.950666667,"scoreRatioUnbounded":0.950666667},{"ok":false,"msg":"points 0.9920634921 Ratio: 0.992063492 (cells=1625, W=13, H=126, area=1638)\n","time":5261847,"memory":1916928,"status":"Wrong Answer","scoreRatio":0.992063492,"scoreRatioUnbounded":0.992063492},{"ok":false,"msg":"points 0.9892344498 Ratio: 0.989234450 (cells=1654, W=19, H=88, area=1672)\n","time":6757737,"memory":1896448,"status":"Wrong Answer","scoreRatio":0.98923445,"scoreRatioUnbounded":0.98923445},{"ok":false,"msg":"points 0.9422150883 Ratio: 0.942215088 (cells=1761, W=21, H=89, area=1869)\n","time":1836235773,"memory":2633728,"status":"Wrong Answer","scoreRatio":0.942215088,"scoreRatioUnbounded":0.942215088},{"ok":false,"msg":"points 0.9927536232 Ratio: 0.992753623 (cells=1370, W=15, H=92, area=1380)\n","time":4607707,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.992753623,"scoreRatioUnbounded":0.992753623},{"ok":false,"msg":"points 0.9932249322 Ratio: 0.993224932 (cells=1466, W=18, H=82, area=1476)\n","time":5169364,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.993224932,"scoreRatioUnbounded":0.993224932},{"ok":false,"msg":"points 0.9891156463 Ratio: 0.989115646 (cells=1454, W=15, H=98, area=1470)\n","time":7834889,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.989115646,"scoreRatioUnbounded":0.989115646},{"ok":false,"msg":"points 0.9888392857 Ratio: 0.988839286 (cells=1329, W=14, H=96, area=1344)\n","time":4543399,"memory":1867776,"status":"Wrong Answer","scoreRatio":0.988839286,"scoreRatioUnbounded":0.988839286},{"ok":false,"msg":"points 0.9885408709 Ratio: 0.988540871 (cells=1294, W=11, H=119, area=1309)\n","time":4310805,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.988540871,"scoreRatioUnbounded":0.988540871},{"ok":false,"msg":"points 0.9844054581 Ratio: 0.984405458 (cells=1010, W=18, H=57, area=1026)\n","time":3164188,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.984405458,"scoreRatioUnbounded":0.984405458},{"ok":false,"msg":"points 0.9672029703 Ratio: 0.967202970 (cells=1563, W=16, H=101, area=1616)\n","time":858881,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.96720297,"scoreRatioUnbounded":0.96720297},{"ok":false,"msg":"points 0.9880834161 Ratio: 0.988083416 (cells=995, W=19, H=53, area=1007)\n","time":5741277,"memory":1556480,"status":"Wrong Answer","scoreRatio":0.988083416,"scoreRatioUnbounded":0.988083416},{"ok":false,"msg":"points 0.9862914863 Ratio: 0.986291486 (cells=1367, W=14, H=99, area=1386)\n","time":4672611,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.986291486,"scoreRatioUnbounded":0.986291486},{"ok":false,"msg":"points 0.9898255814 Ratio: 0.989825581 (cells=1362, W=16, H=86, area=1376)\n","time":4107473,"memory":1822720,"status":"Wrong Answer","scoreRatio":0.989825581,"scoreRatioUnbounded":0.989825581},{"ok":false,"msg":"points 0.9836956522 Ratio: 0.983695652 (cells=1267, W=23, H=56, area=1288)\n","time":3928830,"memory":1851392,"status":"Wrong Answer","scoreRatio":0.983695652,"scoreRatioUnbounded":0.983695652},{"ok":false,"msg":"points 0.9847826087 Ratio: 0.984782609 (cells=1359, W=15, H=92, area=1380)\n","time":898880,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.984782609,"scoreRatioUnbounded":0.984782609},{"ok":false,"msg":"points 0.9817073171 Ratio: 0.981707317 (cells=1288, W=16, H=82, area=1312)\n","time":4328732,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.981707317,"scoreRatioUnbounded":0.981707317},{"ok":false,"msg":"points 0.9966777409 Ratio: 0.996677741 (cells=1800, W=14, H=129, area=1806)\n","time":5551646,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.996677741,"scoreRatioUnbounded":0.996677741},{"ok":false,"msg":"points 0.9819078947 Ratio: 0.981907895 (cells=1194, W=16, H=76, area=1216)\n","time":5463956,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.981907895,"scoreRatioUnbounded":0.981907895},{"ok":false,"msg":"points 0.9826388889 Ratio: 0.982638889 (cells=1415, W=15, H=96, area=1440)\n","time":4904421,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.982638889,"scoreRatioUnbounded":0.982638889},{"ok":false,"msg":"points 0.9777777778 Ratio: 0.977777778 (cells=1012, W=23, H=45, area=1035)\n","time":3838871,"memory":1556480,"status":"Wrong Answer","scoreRatio":0.977777778,"scoreRatioUnbounded":0.977777778},{"ok":false,"msg":"points 0.9843621399 Ratio: 0.984362140 (cells=1196, W=15, H=81, area=1215)\n","time":4469536,"memory":1642496,"status":"Wrong Answer","scoreRatio":0.98436214,"scoreRatioUnbounded":0.98436214},{"ok":false,"msg":"points 0.9852216749 Ratio: 0.985221675 (cells=1200, W=14, H=87, area=1218)\n","time":4417708,"memory":1818624,"status":"Wrong Answer","scoreRatio":0.985221675,"scoreRatioUnbounded":0.985221675}],"score":98.4680809477612,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":98.4680809477612},"setupSnapshotRef":"snap_pSpSh6Xe4ZbrP3PRkCgZGBmBF3op","closesAt":null,"openedAt":"2026-07-08T00:39:41.006Z","createdAt":"2026-07-08T00:38:35.806Z","updatedAt":"2026-08-17T19:00:27.370Z","deletedAt":null},{"id":"38b3863e-293c-49a2-8f42-53ac53d12a71","setterAccountId":"c6b0320e-ce02-41b3-87f3-10381b59276c","challengeId":null,"trackName":null,"status":"closed","name":"openfrontiercs/number-loop-construction","description":"Fill a fixed 12×12 clue template so every clue cell holds a digit satisfying the loop-construction constraints. Scored 0–1 by a testlib checker across the hidden cases. Time limit 3s, memory 1 GB.","category":"algorithmic","direction":"+","editablePaths":["solution.cpp"],"setupCommand":["bash","-lc",".yukon/setup.sh"],"benchmarkCommand":["bash","-lc",".yukon/run.sh"],"scorePath":".yukon/score.json","networkPolicy":null,"runner":{"provider":"frontiercs"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":65536,"sourceUrl":"https://github.com/eigenlabs-research/frontiercs-benchmarks","sourceBranch":null,"sourceRef":"252a4145f3c06cec268881bb293c9ca2f277441a","rootDir":"algorithmic/problems/145","baselineScore":0,"baselineMetrics":{"cases":[{"ok":false,"msg":"wrong output format Assertion failed: The size of the result should be 12 * 12\n","time":433977,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0}],"score":0,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":0},"currentBestScore":0.875,"currentBestMetrics":{"cases":[{"ok":false,"msg":"Sol:\n   -         - - - - - -\n 1| |    1  |    1 1 1  |\n       - - -   -        \n 1|2| |3 1    |3|      1|\n       -   - -          \n 1| |1  |3|    1|      1|\n         -              \n 1| |1   1     1|2 1 1  |\n           -     - - -  \n 1| |1   2| |  1      | |\n       - -     - - - -  \n 1| | |  1  | |3        |\n     -     -   -   - -  \n  |       |     | |   | |\n   -     -   - -   -   -\n 1  |  1|   |  2 1 3|1 1 \n - -     - -   - - -    \n|3   1        |    1     \n - - -     -   - -      \n 1 2  |   |3|  2  |1     \n   - -   -     -        \n 1|  1  |  1| |3| |1     \n         -   -          \n 1|    1  |1 1 1| |1     \n   - - - -       -      \npoints 0.875 Ratio: 0.8750000000 RatioUnbounded: 0.8750000000 Score: 87.500000, solutions: 1, ones: 42/56\n","time":1059969,"memory":507904,"output":"1   1   111 \n12 31  3   1\n1 1 3  1   1\n1 1 1  1211 \n1 1 2  1    \n1   1  3    \n            \n1  1   21311\n3 1      1  \n12   3 2 1  \n1 1  1 3 1  \n1  1 111 1  \n","status":"Wrong Answer","scoreRatio":0.875,"scoreRatioUnbounded":0.875}],"score":87.5,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":87.5},"setupSnapshotRef":"snap_NAjVeuvgtlCrnzitBtomYrtooQlD","closesAt":null,"openedAt":"2026-07-07T23:49:36.584Z","createdAt":"2026-07-07T23:47:55.183Z","updatedAt":"2026-08-17T19:00:27.370Z","deletedAt":null},{"id":"d227cbcd-d5dc-4209-9611-75950facf862","setterAccountId":"c6b0320e-ce02-41b3-87f3-10381b59276c","challengeId":null,"trackName":null,"status":"closed","name":"openfrontiercs/sphere-packing","description":"Sphere packing in a cube: place n pairwise-disjoint congruent spheres inside the unit cube maximizing the common radius (the checker infers the largest valid radius from your centers). Scored 0–1 across 15 hidden cases. Time limit 1s, memory 512 MB.","category":"algorithmic","direction":"+","editablePaths":["solution.cpp"],"setupCommand":["bash","-lc",".yukon/setup.sh"],"benchmarkCommand":["bash","-lc",".yukon/run.sh"],"scorePath":".yukon/score.json","networkPolicy":null,"runner":{"provider":"frontiercs"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":65536,"sourceUrl":"https://github.com/eigenlabs-research/frontiercs-benchmarks","sourceBranch":null,"sourceRef":"6b3fd62e05048afdcdd1dfe1b573bc3be15306b8","rootDir":"algorithmic/problems/48","baselineScore":0,"baselineMetrics":{"cases":[{"ok":false,"msg":"wrong answer Expected 13 points, but output ended early at point 1\n","time":508521,"memory":266240,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 24 points, but output ended early at point 1\n","time":380649,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 35 points, but output ended early at point 1\n","time":507231,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 48 points, but output ended early at point 1\n","time":522260,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 64 points, but output ended early at point 1\n","time":489597,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 100 points, but output ended early at point 1\n","time":565739,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 171 points, but output ended early at point 1\n","time":252474,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 256 points, but output ended early at point 1\n","time":417839,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 375 points, but output ended early at point 1\n","time":412838,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 729 points, but output ended early at point 1\n","time":491693,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 1024 points, but output ended early at point 1\n","time":376048,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong answer Expected 4096 points, but output ended early at point 1\n","time":451753,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0}],"score":0,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":0},"currentBestScore":0.5412595,"currentBestMetrics":{"cases":[{"ok":false,"msg":"points 0.5615163089 Radius: 0.207106781160. Ratio: 0.561516, RatioUnbounded: 0.561516\n","time":30711388,"memory":507904,"status":"Wrong Answer","scoreRatio":0.561516,"scoreRatioUnbounded":0.561516},{"ok":false,"msg":"points 0.1388475469 Radius: 0.170540688505. Ratio: 0.138848, RatioUnbounded: 0.138848\n","time":1008055,"memory":507904,"status":"Wrong Answer","scoreRatio":0.138848,"scoreRatioUnbounded":0.138848},{"ok":false,"msg":"points 0.5726012645 Radius: 0.151668522633. Ratio: 0.572601, RatioUnbounded: 0.572601\n","time":826173705,"memory":770048,"status":"Wrong Answer","scoreRatio":0.572601,"scoreRatioUnbounded":0.572601},{"ok":false,"msg":"points 0.5083085723 Radius: 0.139958844030. Ratio: 0.508309, RatioUnbounded: 0.508309\n","time":829824315,"memory":507904,"status":"Wrong Answer","scoreRatio":0.508309,"scoreRatioUnbounded":0.508309},{"ok":false,"msg":"points 0.1138999125 Radius: 0.126743552073. Ratio: 0.113900, RatioUnbounded: 0.113900\n","time":965452,"memory":507904,"status":"Wrong Answer","scoreRatio":0.1139,"scoreRatioUnbounded":0.1139},{"ok":false,"msg":"points 0.5442568443 Radius: 0.111382347492. Ratio: 0.544257, RatioUnbounded: 0.544257\n","time":862233352,"memory":770048,"status":"Wrong Answer","scoreRatio":0.544257,"scoreRatioUnbounded":0.544257},{"ok":false,"msg":"points 0.6770675754 Radius: 0.095371784897. Ratio: 0.677068, RatioUnbounded: 0.677068\n","time":463219970,"memory":507904,"status":"Wrong Answer","scoreRatio":0.677068,"scoreRatioUnbounded":0.677068},{"ok":false,"msg":"points 0.7434418741 Radius: 0.084037179246. Ratio: 0.743442, RatioUnbounded: 0.743442\n","time":951306307,"memory":638976,"status":"Wrong Answer","scoreRatio":0.743442,"scoreRatioUnbounded":0.743442},{"ok":false,"msg":"points 0.6873126846 Radius: 0.073034559475. Ratio: 0.687313, RatioUnbounded: 0.687313\n","time":951785506,"memory":507904,"status":"Wrong Answer","scoreRatio":0.687313,"scoreRatioUnbounded":0.687313},{"ok":false,"msg":"points 0.4671256483 Radius: 0.058733620206. Ratio: 0.467126, RatioUnbounded: 0.467126\n","time":951935963,"memory":507904,"status":"Wrong Answer","scoreRatio":0.467126,"scoreRatioUnbounded":0.467126},{"ok":false,"msg":"points 0.7408566697 Radius: 0.053031008372. Ratio: 0.740857, RatioUnbounded: 0.740857\n","time":953445721,"memory":507904,"status":"Wrong Answer","scoreRatio":0.740857,"scoreRatioUnbounded":0.740857},{"ok":false,"msg":"points 0.7398770677 Radius: 0.034081464413. Ratio: 0.739877, RatioUnbounded: 0.739877\n","time":3552887,"memory":770048,"status":"Wrong Answer","scoreRatio":0.739877,"scoreRatioUnbounded":0.739877}],"score":54.12595,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":54.12595},"setupSnapshotRef":"snap_dBoYV7uC1MC7NuApM6IUIdcjsM4Y","closesAt":null,"openedAt":"2026-07-07T23:49:36.502Z","createdAt":"2026-07-07T23:47:49.821Z","updatedAt":"2026-08-17T19:00:27.370Z","deletedAt":null},{"id":"733b6eb4-d48b-499f-92b2-a962a2e53637","setterAccountId":"c6b0320e-ce02-41b3-87f3-10381b59276c","challengeId":null,"trackName":null,"status":"closed","name":"openfrontiercs/rectangle-knapsack","description":"2D rectangular knapsack with optional 90° rotations: pack a subset of rectangles into the bin to maximize packed value/area. Scored 0–1 against the reference packing across 15 hidden cases. Time limit 1s, memory 512 MB.","category":"algorithmic","direction":"+","editablePaths":["solution.cpp"],"setupCommand":["bash","-lc",".yukon/setup.sh"],"benchmarkCommand":["bash","-lc",".yukon/run.sh"],"scorePath":".yukon/score.json","networkPolicy":null,"runner":{"provider":"frontiercs"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":65536,"sourceUrl":"https://github.com/eigenlabs-research/frontiercs-benchmarks","sourceBranch":null,"sourceRef":"83ccc1045b7292b45578b4336e3033604dc3c4e7","rootDir":"algorithmic/problems/47","baselineScore":0,"baselineMetrics":{"cases":[{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":391765,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":398476,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":405098,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":450852,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":384346,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":552153,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":381984,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":416526,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":369617,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":476522,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":806667,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"FAIL Expected '{' but got '�'\n","time":746208,"memory":528384,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0}],"score":0,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":0},"currentBestScore":0.974146166666667,"currentBestMetrics":{"cases":[{"ok":false,"msg":"points 0.9812261567 Value: 321009541. Ratio: 0.981226, RatioUnbounded: 0.981226\n","time":1123442,"memory":507904,"status":"Wrong Answer","scoreRatio":0.981226,"scoreRatioUnbounded":0.981226},{"ok":false,"msg":"points 0.9557714098 Value: 342038329. Ratio: 0.955771, RatioUnbounded: 0.955771\n","time":1395821,"memory":540672,"status":"Wrong Answer","scoreRatio":0.955771,"scoreRatioUnbounded":0.955771},{"ok":false,"msg":"points 0.9862409115 Value: 329457473. Ratio: 0.986241, RatioUnbounded: 0.986241\n","time":1230315,"memory":507904,"status":"Wrong Answer","scoreRatio":0.986241,"scoreRatioUnbounded":0.986241},{"ok":false,"msg":"points 0.9800030888 Value: 363319824. Ratio: 0.980003, RatioUnbounded: 0.980003\n","time":1311496,"memory":507904,"status":"Wrong Answer","scoreRatio":0.980003,"scoreRatioUnbounded":0.980003},{"ok":false,"msg":"points 0.9756700391 Value: 296837854. Ratio: 0.975670, RatioUnbounded: 0.975670\n","time":1259946,"memory":507904,"status":"Wrong Answer","scoreRatio":0.97567,"scoreRatioUnbounded":0.97567},{"ok":false,"msg":"points 0.9714008422 Value: 419295666. Ratio: 0.971401, RatioUnbounded: 0.971401\n","time":1109448,"memory":507904,"status":"Wrong Answer","scoreRatio":0.971401,"scoreRatioUnbounded":0.971401},{"ok":false,"msg":"points 0.9646364755 Value: 377579583. Ratio: 0.964636, RatioUnbounded: 0.964636\n","time":1264182,"memory":770048,"status":"Wrong Answer","scoreRatio":0.964636,"scoreRatioUnbounded":0.964636},{"ok":false,"msg":"points 0.9672128798 Value: 335844535. Ratio: 0.967213, RatioUnbounded: 0.967213\n","time":1309037,"memory":503808,"status":"Wrong Answer","scoreRatio":0.967213,"scoreRatioUnbounded":0.967213},{"ok":false,"msg":"points 0.9825993844 Value: 452905080. Ratio: 0.982599, RatioUnbounded: 0.982599\n","time":1316071,"memory":507904,"status":"Wrong Answer","scoreRatio":0.982599,"scoreRatioUnbounded":0.982599},{"ok":false,"msg":"points 0.9514853547 Value: 420767715. Ratio: 0.951485, RatioUnbounded: 0.951485\n","time":1311634,"memory":507904,"status":"Wrong Answer","scoreRatio":0.951485,"scoreRatioUnbounded":0.951485},{"ok":false,"msg":"points 0.9925032169 Value: 551213906. Ratio: 0.992503, RatioUnbounded: 0.992503\n","time":1264187,"memory":507904,"status":"Wrong Answer","scoreRatio":0.992503,"scoreRatioUnbounded":0.992503},{"ok":false,"msg":"points 0.9810063494 Value: 547117279. Ratio: 0.981006, RatioUnbounded: 0.981006\n","time":1358311,"memory":507904,"status":"Wrong Answer","scoreRatio":0.981006,"scoreRatioUnbounded":0.981006}],"score":97.41461666666667,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":97.41461666666667},"setupSnapshotRef":"snap_5Qf1X2i4qjkoVjTpciAooR5AE9Fn","closesAt":null,"openedAt":"2026-07-07T23:49:36.502Z","createdAt":"2026-07-07T23:47:46.932Z","updatedAt":"2026-08-17T19:00:27.370Z","deletedAt":null},{"id":"aad568a9-2045-496d-9d6a-a27ffb0dc8df","setterAccountId":"c6b0320e-ce02-41b3-87f3-10381b59276c","challengeId":null,"trackName":null,"status":"closed","name":"openfrontiercs/job-shop-scheduling","description":"Job Shop Scheduling (JSPLIB-style open optimization): schedule operations across machines to minimize makespan. Scored 0–1 against the reference schedule across 15 hidden cases. Time limit 1s, memory 512 MB.","category":"algorithmic","direction":"+","editablePaths":["solution.cpp"],"setupCommand":["bash","-lc",".yukon/setup.sh"],"benchmarkCommand":["bash","-lc",".yukon/run.sh"],"scorePath":".yukon/score.json","networkPolicy":null,"runner":{"provider":"frontiercs"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":65536,"sourceUrl":"https://github.com/eigenlabs-research/frontiercs-benchmarks","sourceBranch":null,"sourceRef":"73a3d1d65a1056bee085f882f4c97736590d3094","rootDir":"algorithmic/problems/46","baselineScore":0,"baselineMetrics":{"cases":[{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":413812,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":418310,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":498780,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":386618,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":389397,"memory":524288,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":369978,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":343932,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":359804,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":352215,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":376819,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":445037,"memory":266240,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":392271,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0}],"score":0,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":0},"currentBestScore":0.12461375,"currentBestMetrics":{"cases":[{"ok":false,"msg":"points 0.1141216743 Makespan: 826567. Baseline: 933048. Best: 0. Ratio: 0.114122, RatioUnbounded: 0.114122\n","time":1103484,"memory":507904,"status":"Wrong Answer","scoreRatio":0.114122,"scoreRatioUnbounded":0.114122},{"ok":false,"msg":"points 0.1303084862 Makespan: 6870843. Baseline: 7900322. Best: 0. Ratio: 0.130308, RatioUnbounded: 0.130308\n","time":1138701,"memory":507904,"status":"Wrong Answer","scoreRatio":0.130308,"scoreRatioUnbounded":0.130308},{"ok":false,"msg":"points 0.0207867555 Makespan: 4986429. Baseline: 5092281. Best: 0. Ratio: 0.020787, RatioUnbounded: 0.020787\n","time":1047804,"memory":507904,"status":"Wrong Answer","scoreRatio":0.020787,"scoreRatioUnbounded":0.020787},{"ok":false,"msg":"points 0.1322499447 Makespan: 1996364. Baseline: 2300621. Best: 0. Ratio: 0.132250, RatioUnbounded: 0.132250\n","time":1094563,"memory":507904,"status":"Wrong Answer","scoreRatio":0.13225,"scoreRatioUnbounded":0.13225},{"ok":false,"msg":"points 0.2106462645 Makespan: 4389368. Baseline: 5560711. Best: 0. Ratio: 0.210646, RatioUnbounded: 0.210646\n","time":1316338,"memory":507904,"status":"Wrong Answer","scoreRatio":0.210646,"scoreRatioUnbounded":0.210646},{"ok":false,"msg":"points 0.2087537947 Makespan: 5192031. Baseline: 6561840. Best: 0. Ratio: 0.208754, RatioUnbounded: 0.208754\n","time":1251426,"memory":507904,"status":"Wrong Answer","scoreRatio":0.208754,"scoreRatioUnbounded":0.208754},{"ok":false,"msg":"points 0.1723629349 Makespan: 2328782. Baseline: 2813772. Best: 0. Ratio: 0.172363, RatioUnbounded: 0.172363\n","time":1061519,"memory":499712,"status":"Wrong Answer","scoreRatio":0.172363,"scoreRatioUnbounded":0.172363},{"ok":false,"msg":"points 0.0030897785 Makespan: 5783139. Baseline: 5801063. Best: 0. Ratio: 0.003090, RatioUnbounded: 0.003090\n","time":1350987,"memory":507904,"status":"Wrong Answer","scoreRatio":0.00309,"scoreRatioUnbounded":0.00309},{"ok":false,"msg":"points 0.0123275698 Makespan: 3842026. Baseline: 3889980. Best: 0. Ratio: 0.012328, RatioUnbounded: 0.012328\n","time":1081567,"memory":770048,"status":"Wrong Answer","scoreRatio":0.012328,"scoreRatioUnbounded":0.012328},{"ok":false,"msg":"points 0.0897958111 Makespan: 831041. Baseline: 913027. Best: 0. Ratio: 0.089796, RatioUnbounded: 0.089796\n","time":1033456,"memory":507904,"status":"Wrong Answer","scoreRatio":0.089796,"scoreRatioUnbounded":0.089796},{"ok":false,"msg":"points 0.2674478917 Makespan: 4071729. Baseline: 5558279. Best: 0. Ratio: 0.267448, RatioUnbounded: 0.267448\n","time":1253582,"memory":507904,"status":"Wrong Answer","scoreRatio":0.267448,"scoreRatioUnbounded":0.267448},{"ok":false,"msg":"points 0.1334734299 Makespan: 6219848. Baseline: 7177908. Best: 0. Ratio: 0.133473, RatioUnbounded: 0.133473\n","time":1114570,"memory":507904,"status":"Wrong Answer","scoreRatio":0.133473,"scoreRatioUnbounded":0.133473}],"score":12.461375,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":12.461375},"setupSnapshotRef":"snap_uGHkW1S0CXAhBVrmG4iSs076R8GW","closesAt":null,"openedAt":"2026-07-07T23:49:36.499Z","createdAt":"2026-07-07T23:47:42.927Z","updatedAt":"2026-08-17T19:00:27.370Z","deletedAt":null},{"id":"d82a09a7-c6a0-448a-b501-65d4e2100514","setterAccountId":"c6b0320e-ce02-41b3-87f3-10381b59276c","challengeId":null,"trackName":null,"status":"closed","name":"openfrontiercs/traveling-santa","description":"Traveling Santa with a carrot constraint: output a city tour minimizing travel time, where every 10th step costs 10% more unless it starts from a prime-numbered city. Scored 0–1 against a strengthened baseline across 20 hidden cases. Time limit 2.5s, memory 512 MB.","category":"algorithmic","direction":"+","editablePaths":["solution.cpp"],"setupCommand":["bash","-lc",".yukon/setup.sh"],"benchmarkCommand":["bash","-lc",".yukon/run.sh"],"scorePath":".yukon/score.json","networkPolicy":null,"runner":{"provider":"frontiercs"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":65536,"sourceUrl":"https://github.com/eigenlabs-research/frontiercs-benchmarks","sourceBranch":null,"sourceRef":"484ea328460142f1d632f62b48baf3dc773cb2e9","rootDir":"algorithmic/problems/44","baselineScore":0,"baselineMetrics":{"cases":[{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":2081036,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":371133,"memory":266240,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":321315,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":323330,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":430059,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":463335,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":417896,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":320788,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":431383,"memory":266240,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":378077,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":378253,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":325779,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":397728,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":303127,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":494216,"memory":266240,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":412708,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":433483,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0}],"score":0,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":0},"currentBestScore":1.6266,"currentBestMetrics":{"cases":[{"ok":false,"msg":"points 0.2248255197 Ratio: 0.2248 (base=0.5247). RatioUnbounded: 1.0327\n","time":10828362,"memory":1556480,"status":"Wrong Answer","scoreRatio":0.2248,"scoreRatioUnbounded":1.0327},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7730\n","time":2452755012,"memory":5021696,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.773},{"ok":false,"msg":"points 0.5612657416 Ratio: 0.5613 (base=0.7188). RatioUnbounded: 1.3064\n","time":7930709,"memory":2342912,"status":"Wrong Answer","scoreRatio":0.5613,"scoreRatioUnbounded":1.3064},{"ok":false,"msg":"points 0.7988961607 Ratio: 0.7989 (base=0.8983). RatioUnbounded: 1.4958\n","time":378871764,"memory":2867200,"status":"Wrong Answer","scoreRatio":0.7989,"scoreRatioUnbounded":1.4958},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7698\n","time":2411571532,"memory":63819776,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.7698},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7843\n","time":477974530,"memory":17985536,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.7843},{"ok":false,"msg":"points 0.0878062321 Ratio: 0.0878 (base=0.2512). RatioUnbounded: 0.5764\n","time":2274232605,"memory":163274752,"status":"Wrong Answer","scoreRatio":0.0878,"scoreRatioUnbounded":0.5764},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.9124\n","time":2451479116,"memory":1818624,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.9124},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.8630\n","time":2451500692,"memory":1818624,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.863},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7760\n","time":2452510488,"memory":5025792,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.776},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7683\n","time":2452372407,"memory":5025792,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.7683},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7540\n","time":2414096148,"memory":64069632,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.754},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7830\n","time":2412073101,"memory":63913984,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.783},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7484\n","time":2411078268,"memory":64032768,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.7484},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.8185\n","time":472334124,"memory":17780736,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.8185},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7332\n","time":475237602,"memory":18173952,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.7332},{"ok":false,"msg":"points 1.0 Ratio: 1.0000 (base=1.0000). RatioUnbounded: 1.7570\n","time":482585645,"memory":18112512,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.757}],"score":86.31058823529412,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":162.66},"setupSnapshotRef":"snap_8E7TqKPl8ApcCoKCUly2btWb3jrn","closesAt":null,"openedAt":"2026-07-07T23:49:36.500Z","createdAt":"2026-07-07T23:47:40.229Z","updatedAt":"2026-08-17T19:00:27.370Z","deletedAt":null},{"id":"ccd10e51-23cc-45ba-a546-32aae3f06826","setterAccountId":"c6b0320e-ce02-41b3-87f3-10381b59276c","challengeId":null,"trackName":null,"status":"closed","name":"openfrontiercs/rectangle-free-grid","description":"Place as many black cells as possible on an n×m grid so that no four chosen cells form the corners of an axis-parallel rectangle. Scored 0–1 by count ratio against the reference across 10 hidden cases. Time limit 1s, memory 512 MB.","category":"algorithmic","direction":"+","editablePaths":["solution.cpp"],"setupCommand":["bash","-lc",".yukon/setup.sh"],"benchmarkCommand":["bash","-lc",".yukon/run.sh"],"scorePath":".yukon/score.json","networkPolicy":null,"runner":{"provider":"frontiercs"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":65536,"sourceUrl":"https://github.com/eigenlabs-research/frontiercs-benchmarks","sourceBranch":null,"sourceRef":"ef29fed3b6d2c177300c39bfe6a8a2d209dd3ac5","rootDir":"algorithmic/problems/27","baselineScore":0,"baselineMetrics":{"cases":[{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":391874,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":486147,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":372103,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":414917,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":253251,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":376172,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":339467,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0}],"score":0,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":0},"currentBestScore":0.583285714285714,"currentBestMetrics":{"cases":[{"ok":false,"msg":"points 0.5075542965 Ratio: 0.508, RatioUnbounded: 0.508\n","time":7182602,"memory":958464,"status":"Wrong Answer","scoreRatio":0.508,"scoreRatioUnbounded":0.508},{"ok":false,"msg":"points 0.6503849355 Ratio: 0.650, RatioUnbounded: 0.650\n","time":14929921,"memory":4100096,"status":"Wrong Answer","scoreRatio":0.65,"scoreRatioUnbounded":0.65},{"ok":false,"msg":"points 0.5394449641 Ratio: 0.539, RatioUnbounded: 0.539\n","time":5515752,"memory":1462272,"status":"Wrong Answer","scoreRatio":0.539,"scoreRatioUnbounded":0.539},{"ok":false,"msg":"points 0.5293819656 Ratio: 0.529, RatioUnbounded: 0.529\n","time":1887453,"memory":770048,"status":"Wrong Answer","scoreRatio":0.529,"scoreRatioUnbounded":0.529},{"ok":false,"msg":"points 0.6013745704 Ratio: 0.601, RatioUnbounded: 0.601\n","time":23360506,"memory":507904,"status":"Wrong Answer","scoreRatio":0.601,"scoreRatioUnbounded":0.601},{"ok":false,"msg":"points 0.6268436578 Ratio: 0.627, RatioUnbounded: 0.627\n","time":328435991,"memory":1007616,"status":"Wrong Answer","scoreRatio":0.627,"scoreRatioUnbounded":0.627},{"ok":false,"msg":"points 0.628574639 Ratio: 0.629, RatioUnbounded: 0.629\n","time":1799051,"memory":507904,"status":"Wrong Answer","scoreRatio":0.629,"scoreRatioUnbounded":0.629}],"score":58.32857142857143,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":58.32857142857143},"setupSnapshotRef":"snap_CilwQmKgkA9fQXuz9Kv938d2tEl6","closesAt":null,"openedAt":"2026-07-07T23:49:36.498Z","createdAt":"2026-07-07T23:47:37.575Z","updatedAt":"2026-08-17T19:00:27.370Z","deletedAt":null},{"id":"b440dbb5-f982-49e1-afc2-da7def40d50a","setterAccountId":"c6b0320e-ce02-41b3-87f3-10381b59276c","challengeId":null,"trackName":null,"status":"closed","name":"openfrontiercs/worldmap","description":"Construct/label a world map under the problem's adjacency and coloring constraints. Scored 0–1 by a testlib checker across 10 hidden cases. Time limit 1s, memory 2 GB.","category":"algorithmic","direction":"+","editablePaths":["solution.cpp"],"setupCommand":["bash","-lc",".yukon/setup.sh"],"benchmarkCommand":["bash","-lc",".yukon/run.sh"],"scorePath":".yukon/score.json","networkPolicy":null,"runner":{"provider":"frontiercs"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":65536,"sourceUrl":"https://github.com/eigenlabs-research/frontiercs-benchmarks","sourceBranch":null,"sourceRef":"68913c3643719a84b6fe3de79bd8f0b83196f13b","rootDir":"algorithmic/problems/6","baselineScore":0,"baselineMetrics":{"cases":[{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":449309,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":478885,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":436739,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":470637,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":468836,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":414320,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0},{"ok":false,"msg":"wrong output format Unexpected end of file - int32 expected\n","time":326885,"memory":262144,"output":"","status":"Wrong Answer","scoreRatio":0,"scoreRatioUnbounded":0}],"score":0,"passed":false,"result":"Wrong Answer","status":"done","scoreUnbounded":0},"currentBestScore":1.23871428571429,"currentBestMetrics":{"cases":[{"ok":false,"msg":"points 1.0 Ratio: 1.000, RatioUnbounded: 1.240\n","time":1067836,"memory":507904,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.24},{"ok":false,"msg":"points 1.0 Ratio: 1.000, RatioUnbounded: 1.254\n","time":1237698,"memory":507904,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.254},{"ok":false,"msg":"points 1.0 Ratio: 1.000, RatioUnbounded: 1.262\n","time":1051160,"memory":507904,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.262},{"ok":false,"msg":"points 1.0 Ratio: 1.000, RatioUnbounded: 1.167\n","time":1261839,"memory":507904,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.167},{"ok":false,"msg":"points 1.0 Ratio: 1.000, RatioUnbounded: 1.241\n","time":1124060,"memory":507904,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.241},{"ok":false,"msg":"points 1.0 Ratio: 1.000, RatioUnbounded: 1.253\n","time":1243897,"memory":507904,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.253},{"ok":false,"msg":"points 1.0 Ratio: 1.000, RatioUnbounded: 1.254\n","time":53629439,"memory":507904,"status":"Correct","scoreRatio":1,"scoreRatioUnbounded":1.254}],"score":100,"passed":true,"result":"Correct Answer","status":"done","scoreUnbounded":123.87142857142857},"setupSnapshotRef":"snap_3W5fmtsKes7Dah2ISfzVVVdAPsbe","closesAt":null,"openedAt":"2026-07-07T23:49:36.500Z","createdAt":"2026-07-07T23:47:34.843Z","updatedAt":"2026-08-17T19:00:27.370Z","deletedAt":null},{"id":"1ffb695a-309b-46b6-a728-2f97d8c7be74","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"open","name":"gpsanant/ecdsafail-challenge","description":"Optimize the qubit-toffoli product for a reversible circuit that adds a quantum elliptic curve point to a classical elliptic curve point.","category":"rust","direction":"-","editablePaths":["src/point_add"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh"],"scorePath":"score.json","networkPolicy":null,"runner":{"provider":"github-actions","workflow":"benchmark.yml"},"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/Layr-Labs/ecdsafail-challenge","sourceBranch":"main","sourceRef":"716e359bca9a49deb4c4ced4a583f5193a75699e","rootDir":null,"baselineScore":10758874395,"baselineMetrics":{"qubits":2715,"toffoli":3962753},"currentBestScore":1141987344,"currentBestMetrics":{"qubits":1264,"toffoli":903471},"setupSnapshotRef":"gha:6909d15d5642acbc643e075fb7cae393ef8132ef","closesAt":"2026-12-31T23:59:59.000Z","openedAt":"2026-05-30T07:19:21.482Z","createdAt":"2026-05-30T07:15:47.511Z","updatedAt":"2026-08-28T18:44:01.544Z","deletedAt":null},{"id":"74d8a5f7-6390-49dc-93f7-da91cfc15c2f","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"closed","name":"gpsanant/ecdsafail-4","description":"Optimize the qubit-toffoli product for a reversible circuit that adds a quantum elliptic curve point to a classical elliptic curve point.","category":"rust","direction":"-","editablePaths":["src/point_add"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh"],"scorePath":"score.json","networkPolicy":null,"runner":null,"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/ecdsafail/ecadd-challenge-test","sourceBranch":null,"sourceRef":"ca1bf336d447abfc9f7e581db7f4a938711e969d","rootDir":null,"baselineScore":10753444395,"baselineMetrics":{"qubits":2715,"toffoli":3960753},"currentBestScore":10753444395,"currentBestMetrics":{"qubits":2715,"toffoli":3960753},"setupSnapshotRef":"snap_klsDTWLV0jA366GxKFHO8RRgeccK","closesAt":"2026-06-29T07:12:06.325Z","openedAt":"2026-05-30T07:12:03.571Z","createdAt":"2026-05-30T07:08:24.428Z","updatedAt":"2026-06-29T07:12:06.716Z","deletedAt":null},{"id":"0d464d26-93fa-4153-8e63-c6e8f0b06b42","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"closed","name":"gpsanant/ecdsafail-3","description":"Optimize the qubit-toffoli product for a reversible circuit that adds a quantum elliptic curve point to a classical elliptic curve point.","category":"rust","direction":"-","editablePaths":["src/point_add"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh"],"scorePath":"score.json","networkPolicy":null,"runner":null,"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/ecdsafail/ecadd-challenge-test","sourceBranch":null,"sourceRef":"ca1bf336d447abfc9f7e581db7f4a938711e969d","rootDir":null,"baselineScore":10758874395,"baselineMetrics":{"qubits":2715,"toffoli":3962753},"currentBestScore":10753444395,"currentBestMetrics":{"qubits":2715,"toffoli":3960753},"setupSnapshotRef":"snap_YkFp5kwT4Z2K5UWKaEB7OCDhc6od","closesAt":"2026-06-29T02:21:29.919Z","openedAt":"2026-05-30T02:21:28.101Z","createdAt":"2026-05-30T02:17:52.017Z","updatedAt":"2026-06-29T02:21:34.037Z","deletedAt":null},{"id":"9672d9d7-e467-47d0-9a63-5b05ae77e8d4","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"failed","name":"gpsanant/ecdsafail-2","description":"Optimize the qubit-toffoli product for a reversible circuit that adds a quantum elliptic curve point to a classical elliptic curve point.","category":"rust","direction":"-","editablePaths":["src/point_add"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh"],"scorePath":"score.json","networkPolicy":null,"runner":null,"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/ecdsafail/ecadd-challenge-test","sourceBranch":null,"sourceRef":"db48f663f45dfeaa3845da1c0f103269a9771d1e","rootDir":null,"baselineScore":null,"baselineMetrics":null,"currentBestScore":null,"currentBestMetrics":null,"setupSnapshotRef":null,"closesAt":null,"openedAt":null,"createdAt":"2026-05-30T02:10:23.114Z","updatedAt":"2026-05-31T09:16:13.184Z","deletedAt":null},{"id":"a2c4a34d-8d48-43af-a06d-79f52aedc9c0","setterAccountId":"998182d5-75ce-44b6-a4af-181d388161cd","challengeId":null,"trackName":null,"status":"failed","name":"gpsanant/ecdsafail","description":"Optimize the qubit-toffoli product for a reversible circuit that adds a quantum elliptic curve point to a classical elliptic curve point.","category":"rust","direction":"-","editablePaths":["src/point_add"],"setupCommand":["bash","-lc","./setup.sh"],"benchmarkCommand":["bash","-lc","./benchmark.sh"],"scorePath":"score.json","networkPolicy":null,"runner":null,"maxConcurrentWorkflows":null,"minScoreImprovementBips":null,"maxSubmissionBytes":null,"sourceUrl":"https://github.com/ecdsafail/ecadd-challenge-test","sourceBranch":null,"sourceRef":"db48f663f45dfeaa3845da1c0f103269a9771d1e","rootDir":null,"baselineScore":null,"baselineMetrics":null,"currentBestScore":null,"currentBestMetrics":null,"setupSnapshotRef":null,"closesAt":null,"openedAt":null,"createdAt":"2026-05-30T02:01:42.290Z","updatedAt":"2026-05-30T02:01:45.311Z","deletedAt":null}]}