lol

they just have the wrong culture for AI

@Dulaman I can run using a M3 Ultra basically any model I want using mlx and they also have it almost within a single day after a novel model release.

Ok I lose.
Here it is, plainly:

I am correct technically
I am correct mathematically
I am correct architecturally.

I am correct empirically.

And none of that matters if I don't control the definition of AI

@Kearm20

Resolves yes if Apple has an AI tool widely considered top tier at any time by end of year 2025.

I am more flexible on the definition of AI then I am on common acceptance.

@GCS I don't want to be like pedantic but what is the definition of widely accepted because professional widely accepted and personally general public is very different. Most people use what I say accidentally just because it's built into the OS.

Is there a definition of widely accepted you think that your example falls under?

For something to be widely accepted I would expect there to be some reporting/blogging/discussion about it.

Assuming you are correct, it seems like you just reverse engineered this yesterday and otherwise it's a secret.

@GCS September 2024 implementation. Presentation at WDC June 9, 2025.https://developer.apple.com/videos/play/wwdc2025/300/ it's frequently used by Applev devs https://developer.apple.com/videos/play/wwdc2024/10177/

Enhance your app with machine-learning-based video effects - WWDC25 - Videos - Apple Developer

Discover how to add effects like frame rate conversion, super resolution, and noise filtering to improve video editing and live streaming...

The thing is it's standard not "secret".

https://developer.apple.com/videos/play/wwdc2024/10177/ ananother Apple WWDC talk on it

Use HDR for dynamic image experiences in your app - WWDC24 - Videos - Apple Developer

Discover how to read and write HDR images and process HDR content in your app. Explore the new supported HDR image formats and advanced...

https://trac.ffmpeg.org/wiki/HWAccelIntro#VideoToolbox

It's exposed in FFmpeg's documentation.

HWAccelIntro – FFmpeg

@GCS Here is a script I made using VideoToolbox as AI SOTA
```
#!/bin/bash

set -e

# --- CONFIG ---

INPUT_VIDEO="$1"

OUTPUT_TEXT="${2:-output_ocr.txt}"

RAM_DISK_NAME="M4_OCR_Scratch"

RAM_DISK_GB=64 # Generous buffer for 128GB Unified Memory

# --------------

if [ -z "$INPUT_VIDEO" ]; then

echo "Usage: ./run_pipeline.sh <video_file> [output_text_file]"

exit 1

fi

# 1. SETUP RAM DISK

SECTORS=$(($RAM_DISK_GB 1024 1024 * 2))

echo "💾 Creating ${RAM_DISK_GB}GB RAM Disk..."

# Create RAM disk and trim whitespace from output

RAM_DISK_PATH=$(hdiutil attach -nomount ram://$SECTORS | tr -d '[:space:]')

if [ -z "$RAM_DISK_PATH" ]; then

echo "❌ Failed to create RAM disk"

exit 1

fi

echo "📀 RAM disk created at: $RAM_DISK_PATH"

# Format the RAM disk

if ! diskutil erasevolume HFS+ "$RAM_DISK_NAME" "$RAM_DISK_PATH"; then

echo "❌ Failed to format RAM disk. Cleaning up..."

hdiutil detach "$RAM_DISK_PATH" -force 2>/dev/null || true

exit 1

fi

MOUNT_POINT="/Volumes/$RAM_DISK_NAME"

# Cleanup trap

cleanup() {

echo "🧹 Ejecting RAM Disk..."

hdiutil detach "$RAM_DISK_PATH" -force > /dev/null

}

trap cleanup EXIT

# 2. EXTRACT FRAMES

echo "🎞️ Extracting frames using VideoToolbox..."

# Using jpg for speed/space, q:v 2 is high quality enough for OCR

ffmpeg -hwaccel videotoolbox -i "$INPUT_VIDEO" \

-vsync 0 -q:v 2 "$MOUNT_POINT/frame_%04d.jpg" \

-hide_banner -loglevel error -stats

# 3. RUN OCR

echo "🧠 Starting DeepSeek-OCR..."

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

"$SCRIPT_DIR/.venv/bin/python" "$SCRIPT_DIR/ocr_worker.py" "$MOUNT_POINT" "$OUTPUT_TEXT"

echo "✅ Done! Results saved to $OUTPUT_TEXT"
```

@Kearm20

🎞️ Extracting frames using VideoToolbox...

frame= 431 fps=0.0 q=2.0 size=N/A time=00:00:08.03 bitrate=N/A speed=15.9x elapsed=0:00:00.50

frame= 994 fps=987 q=2.0 size=N/A time=00:00:18.15 bitrate=N/A speed= 18x elapsed=0:00:01.00

frame= 1572 fps=1040 q=2.0 size=N/A time=00:00:28.67 bitrate=N/A speed= 19x elapsed=0:00:01.51

frame= 2129 fps=1057 q=2.0 size=N/A time=00:00:38.83 bitrate=N/A speed=19.3x elapsed=0:00:02.01

frame= 2691 fps=1068 q=2.0 size=N/A time=00:00:49.02 bitrate=N/A speed=19.5x elapsed=0:00:02.51

frame= 3232 fps=1069 q=2.0 size=N/A time=00:00:59.51 bitrate=N/A speed=19.7x elapsed=0:00:03.02

frame= 3797 fps=1076 q=2.0 size=N/A time=00:01:09.89 bitrate=N/A speed=19.8x elapsed=0:00:03.52

frame= 4358 fps=1080 q=2.0 size=N/A time=00:01:20.34 bitrate=N/A speed=19.9x elapsed=0:00:04.03

frame= 4903 fps=1080 q=2.0 size=N/A time=00:01:30.50 bitrate=N/A speed=19.9x elapsed=0:00:04.53

frame= 5470 fps=1085 q=2.0 size=N/A time=00:01:40.71 bitrate=N/A speed= 20x elapsed=0:00:05.04

frame= 5548 fps=1091 q=2.0 Lsize=N/A time=00:01:42.42 bitrate=N/A speed=20.1x elapsed=0:00:05.08

The speed is absurd

Currently doing GPU traces so that I can prove its an AI model.

@Kearm20 Stop posting AI slop. pls

@Pazzaz WHY? It's proof?

@Pazzaz Manifold then posts "AI slop" Update 2025-12-29 (PST) (AI summary of creator comment): The creator is not convinced that Apple's VideoToolbox/VTHDRPerFrameMetadataGenerationSession qualifies as:

• An "AI" tool in the modern sense

• Best in class compared to alternatives like MediaCodec or other APIs

The creator considers this an older video processing technology rather than a top-tier AI tool for resolution purposes.

I also hardly think manually doing GPU tracing is "AI slop" despite the script being AI written.

@GCS This is resolving as true based on this tool.
https://developer.apple.com/documentation/videotoolbox/vthdrperframemetadatagenerationsession-api-collection?language=objc
VTHDRPerFrameMetadataGenerationSession is Apple's real-time, ML-based HDR metadata inference system deployed globally across iPhone, iPad, Mac, and Apple TV.

Why it's top class:

1. Solves what others avoid — Perceptual HDR inference from ambiguous content, inline at video rate. Others go offline or don't attempt it.

2. Constraint stack no one else clears — Real-time latency, fixed power/thermal, causal operation, zero instability, standards compliance, cross-gen compatibility—simultaneously.

3. Vertical integration advantage — Privileged access to pixel buffers, display characterization, hardware tone-mapping, encoder internals, temporal history, OS scheduling. Unreplicable outside Apple's stack.

4. Bounded learned behavior — Non-deterministic inference constrained by HDR standards. Adaptation without chaos; generational improvement without regression.

5. Best in class for its regime — Nothing else is automatic + real-time + per-frame + consumer-scale + OS-integrated + metadata-driven HDR. The category has one occupant.
   

@Kearm20 There's actually a couple examples of this.
VideoToolbox treats media processing as a real-time AI control system rather than a batch encoding problem.

It solves problems others avoided, including deterministic decode behavior across devices, and zero-copy memory-safe media pipelines.

The framework dynamically retunes encoders and coordinates scheduling with ML workloads to maintain stability under strict latency, power, and thermal constraints.

Its learned, non-deterministic models operate within tightly bounded standards to ensure perceptual consistency and system safety at consumer scale.

This architecture enables Apple to deploy invisible, always-on AI media intelligence that no other platform has matched in scope or reliability.

@Kearm20 I don’t see any evidence how this is even widely considered as an “AI tool”

@Kearm20

Yeah I'm not convinced. This is an older video processing technology. You'd need to convince me this is "AI" in the modern sense and best in class compared to MediaCodec or some other API

@GCS https://drive.google.com/file/d/1J0JQrlJWm6y5nrnw41poM-DACCzTNbnP/view?usp=sharing

Here is my proof.

Apple_AI_Analysis.pdf

@Kearm20 Functionally, this kind of systematic translation in real time requires a learned AI system.

@Kearm20

I like there you are going with this but I'm still not convinced.

You are reverse engineering some features and making a claim that I can not independently confirm. Apple doesn't have any documentation on their implementation that I can find. I don't see how this could possibly solve yes for "widely considered top tier" if it's a secret no one talks about.

You'll need more to snipe a 2 year old market a day before close.

@GCS I'm pulling the model. I have SIP disabled MBP so I can in thoery.