Particle Effect Video Workflow

By @Pluvio9yte. Made with Claude Opus 5.5. Watch the original post.

Prompt

Sharing my production workflow publicly: send it to Opus5.5 and it's ready to use

  1. Choose the footage

  Three criteria, and they directly decide success or failure:

  - Complete subject: the person or object is fully in frame. Details get lost in processing, and viewers recognize things by their outline, so a close-up turns into a blob.
  - Clean background: a solid color, snow or sky all work, so the subject separates easily from the background.
  - Clear action moments: hits, jumps, landings, turns. These are the anchor points for the effects later on.

  Sources can be footage you shot yourself, clips generated by a video model, or licensed existing clips.

  2. Break down the footage

  Extract three kinds of information from every frame:

  - Color: the original image.
  - Depth: how far each point is from the camera, computed with an off-the-shelf depth estimation model.
  - Subject area: which parts are the subject and which are the background.

  Also find the moments where the shot changes. The key to this step is stability: depth and subject area must be processed consistently within each shot, otherwise playback will jitter.

  3. Decide the "particles" and "styles"

  First decide what you will rebuild the image with: dots, squares, text or lines all work.

  Then design several styles. Each style is really just a different combination of three rules:

  - Where they go: across the whole frame, or only on the subject.
  - How big: one uniform size, or varying with brightness and depth.
  - What color: the original image's colors, or one or two uniform colors.

  Plus one switch: laid flat on a single plane, or pushed into 3D space by depth. Only once they are pushed into 3D can the camera orbit.

  Four to six clearly different styles are enough. You have to tune the particle density by eye: too large and you cannot recognize the content, too small and it looks no different from the original footage.

  4. Find the rhythm

  - Set the tempo: pick the music's tempo and slice time into equal-length cells. Make one cell exactly a whole number of frames, so every switch later on lines up easily.
  - Find the action moments: combine two signals, "the sound suddenly gets louder" and "the image suddenly changes", to find the hit points in the footage.
  - Prepare the music: you can use an existing track and analyze its beat, or synthesize a piece at the chosen tempo and add accents on the hit points.

  5. Arrange

  Make a timetable that states which style, which color scheme and which camera position each cell uses. Keep this table in its own file; later, to adjust the rhythm, you only edit it.

  Four principles for the arrangement:

  - Establish the change at the start: show the original image first, then transition to particles, so viewers understand what is happening.
  - Alternate fast and slow: if it is fast all the way through, viewers go numb; there needs to be a clearly slower stretch in the middle.
  - Clashing colors: the bigger the contrast in color and style between two adjacent cells, the more powerful the switch.
  - The ending needs closure: for example, the particles grow bigger and bigger until they become blocks of color, or it returns to the original image.

  6. Feedback at key moments

  Layer short, sharp reactions on the hit points, graded by intensity:

  - Light feedback on every hit: the particles deform briefly, the camera bumps.
  - Medium feedback only on heavier hits: some of the particles burst toward the camera.
  - Big feedback used only on the one or two heaviest hits: time freezes and the camera orbits.

  The principle is weak and short. Move only some of the particles and finish within a few frames; otherwise the image explodes into a mess and viewers can no longer see what the subject is doing.

  Working methods for the whole process

  - Samples first, full film after: each time you finish a stage, render a few frames to check the result, and only move on once it is confirmed. Render the full film only at the very end.
  - For parameters you are unsure about, produce a whole row at once: for density, for example, have the AI output five or six levels side by side, and you pick.
  - Re-render only what you change: when you adjust one section, re-render only that section.
  - Make the AI look at the images itself: at every step, require it to open the render results and look at them before reporting.
  - Fix all randomness: make sure the same settings give the same render every time; otherwise you cannot make changes step by step.

Browse all 1008 videos