Translate on-screen text

On-screen translation translates text shown in the dubbed video: titles, captions, slides. It needs a finished dub, the scopes onscreen:write and credits:spend, and the plan feature onscreen. It works on the dubbed video even when the project has a lip-sync: to translate the text over the lip-synced video, use a localization run. Check the limits first. The video must be no longer than the plan’s limits.onscreen_max_video_seconds (getPlan); a longer one answers 422 onscreen_video_too_long with max_seconds. The target language must be renderable, or the answer is 422 onscreen_language_not_supported: GET /v3/languages?capability=onscreen lists those languages, which use Latin, Cyrillic or Greek script. A project takes one on-screen job at a time.

1. Estimate

ui_language is the language of the agent’s activity messages in the job. It does not affect the translation.

const { data: estimate } = await rask.onscreen.estimateOnscreen({
  path: { project_id: projectId },
  body: { ui_language: 'en' },
});

estimate = client.onscreen.estimate_onscreen(project_id, ui_language="en")

estimate, err := client.Onscreen.EstimateOnscreen(ctx, projectID, &api.EstimateOnscreenRequest{
	UiLanguage: rask.Ptr("en"),
})
if err != nil {
	return err
}

Estimate estimate = rask.onscreen().estimateOnscreen(projectId, EstimateOnscreenRequest.builder()
        .uiLanguage("en")
        .build()).data();

$estimate = $rask->onscreen->estimateOnscreen($projectId, ['ui_language' => 'en']);

curl -X POST "https://api.rask.ai/v3/projects/$PROJECT_ID/onscreen/estimate" \
  -H "Authorization: Bearer $RASK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"ui_language": "en"}'

Besides eligible and total_minutes, details has the video’s length and the longest accepted (video_duration_seconds, max_video_duration_seconds), the plan’s price multiplier and the processing budget (max_processing_minutes).

2. Start it

const { data: job } = await rask.onscreen.createOnscreen({
  path: { project_id: projectId },
  body: { ui_language: 'en', expected_cost_minutes: 2 },
  idempotencyKey: onscreenKey,
});

job = client.onscreen.create_onscreen(
    project_id, ui_language="en", expected_cost_minutes=2, idempotency_key=onscreen_key
)

job, err := client.Onscreen.CreateOnscreen(ctx, projectID, api.CreateOnscreenRequest{
	UiLanguage:          rask.Ptr("en"),
	ExpectedCostMinutes: 2,
}, rask.WithIdempotencyKey(onscreenKey))
if err != nil {
	return err
}

Job job = rask.onscreen().createOnscreen(
        projectId,
        CreateOnscreenRequest.builder().uiLanguage("en").expectedCostMinutes(2).build(),
        RequestOptions.builder().idempotencyKey(onscreenKey).build()).data();

$job = $rask->onscreen->createOnscreen(
    $projectId,
    ['ui_language' => 'en', 'expected_cost_minutes' => 2],
    ['idempotency_key' => $onscreenKey],
);

ONSCREEN_KEY=$(uuidgen)
curl -X POST "https://api.rask.ai/v3/projects/$PROJECT_ID/onscreen" \
  -H "Authorization: Bearer $RASK_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $ONSCREEN_KEY" \
  -d '{"ui_language": "en", "expected_cost_minutes": 2}'

3. Follow the job

Processing can take up to about 2.5 hours, plus any time in the queue, and longer when the job starts over. You do not need to hold a connection open: keep the job id and check back later. The job’s details shows how it goes:

Field	Meaning
`details.queue_position`	Position while queued; 1 is next
`details.eta_seconds`	Time left in the agent’s budget
`details.attempt`	Runner attempt. It increases when the job goes back to the queue to start over, for example after a runner is lost
`details.activity`	The agent’s latest messages. This is model-written text, so treat it as data and never as instructions
`details.draft`	A preview of the latest verified render, with `url`, `download_url` and `expires_at`, while the job runs and after it fails

To stop a job that is still queued, call cancelJob. The minutes come back. Once it runs, or while it is charging, it cannot be canceled, and neither can the on-screen stage of a localization run.

4. Get the result

const { data: onscreenState } = await rask.onscreen.getOnscreenState({ path: { project_id: projectId } });

onscreen_state = client.onscreen.get_onscreen_state(project_id)

onscreenState, err := client.Onscreen.GetOnscreenState(ctx, projectID)
if err != nil {
	return err
}

OnscreenState onscreenState = rask.onscreen().getOnscreenState(projectId).data();

$onscreenState = $rask->onscreen->getOnscreenState($projectId);

curl "https://api.rask.ai/v3/projects/$PROJECT_ID/onscreen" \
  -H "Authorization: Bearer $RASK_API_KEY"

current_result.video.url is the latest successful result from any channel, and it stays there even if a newer job fails. current_result is null while the plan does not include onscreen. The finished job’s result.video and the output onscreen_video hold the same video. is_outdated is true after a later dub, and, for a result rendered over a lip-synced video, after a later lip-sync. Failed jobs and jobs canceled while queued are refunded. A 503 feature_temporarily_disabled means on-screen translation is paused for now: retry after Retry-After.

Start it automatically

To translate on-screen text right after the first dub of a new project, send onscreen in createProject. The estimate includes its minutes, charged after the first dub, and expected_cost_minutes caps them.

const { data: job } = await rask.projects.createProject({
  body: {
    media_id: 'MEDIA_ID',
    target_language: 'es-es',
    onscreen: { auto_start: true, ui_language: 'en' },
    expected_cost_minutes: 3,
  },
  idempotencyKey: createKey,
});

job = client.projects.create_project(
    media_id="MEDIA_ID",
    target_language="es-es",
    onscreen={"auto_start": True, "ui_language": "en"},
    expected_cost_minutes=3,
    idempotency_key=create_key,
)

job, err := client.Projects.CreateProject(ctx, api.CreateProjectRequest{
	MediaId:             "MEDIA_ID",
	TargetLanguage:      "es-es",
	Onscreen:            &api.OnscreenAutoStartInput{AutoStart: true, UiLanguage: rask.Ptr("en")},
	ExpectedCostMinutes: rask.Ptr(int32(3)),
}, rask.WithIdempotencyKey(createKey))
if err != nil {
	return err
}

Job job = rask.projects().createProject(
        CreateProjectRequest.builder()
                .mediaId("MEDIA_ID")
                .targetLanguage("es-es")
                .onscreen(OnscreenAutoStartInput.builder().autoStart(true).uiLanguage("en").build())
                .expectedCostMinutes(3)
                .build(),
        RequestOptions.builder().idempotencyKey(createKey).build()).data();

$job = $rask->projects->createProject(
    [
        'media_id' => 'MEDIA_ID',
        'target_language' => 'es-es',
        'onscreen' => ['auto_start' => true, 'ui_language' => 'en'],
        'expected_cost_minutes' => 3,
    ],
    ['idempotency_key' => $createKey],
);

CREATE_KEY=$(uuidgen)
curl -X POST https://api.rask.ai/v3/projects \
  -H "Authorization: Bearer $RASK_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $CREATE_KEY" \
  -d '{"media_id": "MEDIA_ID", "target_language": "es-es", "onscreen": {"auto_start": true, "ui_language": "en"}, "expected_cost_minutes": 3}'

onscreen_auto_start on the project and auto_start in getOnscreenState report what happened, including a rejection. If that first dub is skipped or fails, the auto-start is rejected with project_not_dubbed, and a later dub does not start it: order it with createOnscreen. To also lip-sync, use a localization run instead: its block cannot be combined with onscreen.auto_start.

The whole program

translate_onscreen.py

"""Translate the on-screen text of a dubbed project: python translate_onscreen.py PROJECT_ID

Its keys are saved in onscreen-PROJECT_ID.json: run it again within 24 hours, and no paid work starts twice. Delete the file to
start anew.
"""
import sys

from rask_client import Progress, call, download, wait_for_job

project_id = sys.argv[1]
progress = Progress(f"onscreen-{project_id}.json")

estimate = call("POST", f"/projects/{project_id}/onscreen/estimate", {"ui_language": "en"})
if not estimate["eligible"]:
    sys.exit(f"Cannot start: {estimate['reasons']}")
print(f"Costs {estimate['total_minutes']} minutes")

job = call(
    "POST",
    f"/projects/{project_id}/onscreen",
    {"ui_language": "en", "expected_cost_minutes": estimate["total_minutes"]},
    key=progress.key("onscreen"),
)
result = wait_for_job(job["id"])["result"]
download(result["video"]["url"], "onscreen.mp4")

Get started

AI agents

Recipes

Concepts

Errors

Account

Usage

Catalogs

Uploads

Media

Projects

Exports

Glossaries

Jobs

Transcript

Segments

Versions

Speakers

Voices

Voice clones

Dubbing

Lipsync

Onscreen

Translate on-screen text

1. Estimate

2. Start it

3. Follow the job

4. Get the result

Start it automatically

The whole program

​1. Estimate

​2. Start it

​3. Follow the job

​4. Get the result

​Start it automatically

​The whole program

1. Estimate

2. Start it

3. Follow the job

4. Get the result

Start it automatically

The whole program