JoyAI-Video-Edit
Video editing has always meant handing over the clip and waiting. JD just released JoyAI Video Edit, and it changes a video while the video is still playing. The weights are open under Apache two point oh, free for anyone to download and run.
You write the instruction, the model edits the footage
JoyAI Video Edit takes two things: footage, and a written instruction. The model hands back the same footage with that change made. The footage can be a video file, or a live camera feed. The instruction is an ordinary sentence, like turn the video into a watercolor wash style. The model will swap a background, restyle a whole scene, add an object, or take one out.
The edit starts before the video has an ending
A normal AI video editor reads the whole clip before it changes anything, because it looks at how a scene ends to decide how to handle the beginning. JoyAI Video Edit works only from frames that already went by, so the model can begin on a live feed the moment the first frames arrive. JoyAI runs at about thirty frames a second, the rate live video plays at, on one Nvidia B two hundred, a data center graphics chip.
The model edits eight frames at a time, looking only at what already happened
How does a model edit a video it has not finished watching? JoyAI Video Edit works in batches of eight frames. Inside a batch, all eight frames get edited together, so the change is consistent across them. From one batch to the next, the model looks only backward. A live stream can run forever, and holding every past batch would eventually fill the chip’s memory. So the model keeps only the last few batches, plus the very first one. That first batch is the anchor. It stops the edit from sliding away from what the scene looked like when it started.
The model cleans up each frame in two passes instead of dozens
Working in batches alone is still too slow. The usual way an AI makes a picture is to start from static and clean it up, over and over, dozens of passes for a single frame. That will never keep up with live video. So JD trained a fast copy of the model that does the whole job in two passes. JD also trained that copy to check its work against the matching frame of the original video, every batch. The check stops a long edit from drifting away from the source footage. Two passes is what makes the arithmetic work. One batch of eight frames takes two hundred and sixty-six milliseconds, and the two editing passes are a hundred and eighty-five of those milliseconds. Eight frames every two hundred and sixty-six milliseconds is thirty frames a second.
JoyAI tops JD's one-minute editing test and runs at thirty frames a second
JoyAI tops JD’s one-minute editing test and runs at thirty frames a second
JD built its own test for one-minute clips: two hundred and twenty-nine editing tasks across five kinds of edit. JD scored the results itself. JoyAI came first in all five kinds, with an overall score of three point three out of five. The next best live editor scored one point seven one. JoyAI was also the fastest, at thirty point one nine frames a second on video seven hundred and twenty pixels wide, forty to a hundred percent quicker than the other four.
JoyAI takes the vote against all four live editors
JD also asked people to pick between pairs of clips. Against the four other live editors, people picked JoyAI eight or nine times out of ten. Then JD ran JoyAI against editors that get the whole clip first. Against the best open-source one, JoyAI draws, forty-eight percent of the votes against forty-four. Against Kling three point oh Omni and Seedance two point oh, JoyAI loses, about thirty-five percent of the votes against fifty-six. Live editing has caught the open field. Those two still win.
The weights are open. The thirty frames a second still needs a data center chip.
So editing a video no longer has to mean handing it over and waiting. Now editing can happen while the video plays, because the model works eight frames at a time, looks only backward, and cleans up each batch in two passes. Today that takes one data center chip. A graphics card a person could buy is on JD’s own to-do list, still unchecked.
