Skip to content

Commit c35a7b6

Browse files
keighrimkeighrim
authored
Merge pull request #230 from clamsproject/register/0-swt-detection.v8.4
App Submitted - swt-detection.v8.4
2 parents 5f80515 + fdae089 commit c35a7b6

File tree

6 files changed

+470
-79
lines changed

6 files changed

+470
-79
lines changed

‎docs/_apps/swt-detection/index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ title: swt-detection
55
date: 1970-01-01T00:00:00+00:00
66
---
77
Detects scenes with text, like slates, chyrons and credits. This app can run in three modes, depending on `useClassifier`, `useStitcher` parameters. When `useClassifier=True`, it runs in the "TimePoint mode" and generates TimePoint annotations. When `useStitcher=True`, it runs in the "TimeFrame mode" and generates TimeFrame annotations based on existing TimePoint annotations -- if no TimePoint is found, it produces an error. By default, it runs in the 'both' mode and first generates TimePoint annotations and then TimeFrame annotations on them.
8+
- [v8.4](v8.4) ([`@keighrim`](https://github.com/keighrim))
89
- [v8.3](v8.3) ([`@keighrim`](https://github.com/keighrim))
910
- [v8.2](v8.2) ([`@keighrim`](https://github.com/keighrim))
1011
- [v8.1](v8.1) ([`@keighrim`](https://github.com/keighrim))

‎docs/_apps/swt-detection/v8.4/index.md‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
---
2+
layout: posts
3+
classes: wide
4+
title: "Scenes-with-text Detection (v8.4)"
5+
date: 2025-12-14T01:08:09+00:00
6+
---
7+
## About this version
8+
9+
- Submitter: [keighrim](https://github.com/keighrim)
10+
- Submission Time: 2025-12-14T01:08:09+00:00
11+
- Prebuilt Container Image: [ghcr.io/clamsproject/app-swt-detection:v8.4](https://github.com/clamsproject/app-swt-detection/pkgs/container/app-swt-detection/v8.4)
12+
- Release Notes
13+
14+
(no notes provided by the developer)
15+
16+
## About this app (See raw [metadata.json](metadata.json))
17+
18+
**Detects scenes with text, like slates, chyrons and credits. This app can run in three modes, depending on `useClassifier`, `useStitcher` parameters. When `useClassifier=True`, it runs in the "TimePoint mode" and generates TimePoint annotations. When `useStitcher=True`, it runs in the "TimeFrame mode" and generates TimeFrame annotations based on existing TimePoint annotations -- if no TimePoint is found, it produces an error. By default, it runs in the 'both' mode and first generates TimePoint annotations and then TimeFrame annotations on them.**
19+
20+
- App ID: [http://apps.clams.ai/swt-detection/v8.4](http://apps.clams.ai/swt-detection/v8.4)
21+
- App License: Apache 2.0
22+
- Source Repository: [https://github.com/clamsproject/app-swt-detection](https://github.com/clamsproject/app-swt-detection) ([source tree of the submitted version](https://github.com/clamsproject/app-swt-detection/tree/v8.4))
23+
24+
25+
#### Inputs
26+
(**Note**: "*" as a property value means that the property is required but can be any value.)
27+
28+
- [http://mmif.clams.ai/vocabulary/VideoDocument/v1](http://mmif.clams.ai/vocabulary/VideoDocument/v1) (required)
29+
(of any properties)
30+
31+
32+
33+
#### Configurable Parameters
34+
(**Note**: _Multivalued_ means the parameter can have one or more values.)
35+
36+
- `useClassifier`: optional, defaults to `true`
37+
38+
- Type: boolean
39+
- Multivalued: False
40+
- Choices: `false`, **_`true`_**
41+
42+
43+
> Use the image classifier model to generate TimePoint annotations.
44+
- `tpModelName`: optional, defaults to `convnextv2_tiny`
45+
46+
- Type: string
47+
- Multivalued: False
48+
- Choices: `convnextv2_large`, **_`convnextv2_tiny`_**
49+
50+
51+
> Model name to use for classification, only applies when `useClassifier=true`.
52+
- `tpModelBatchSize`: optional, defaults to `200`
53+
54+
- Type: integer
55+
- Multivalued: False
56+
57+
58+
> Number of images to process in a batch for classification. Smaller batch sizes will use less memory but may be slower. The default value of 200 is set to be the safely maximum size for "large" model running on desktop-grade GPU (12GB VRAM). Only applies when `useClassifier=true`.
59+
- `tpUsePosModel`: optional, defaults to `true`
60+
61+
- Type: boolean
62+
- Multivalued: False
63+
- Choices: `false`, **_`true`_**
64+
65+
66+
> Use the model trained with positional features, only applies when `useClassifier=true`.
67+
- `tpStartAt`: optional, defaults to `0`
68+
69+
- Type: integer
70+
- Multivalued: False
71+
72+
73+
> Number of milliseconds into the video to start processing, only applies when `useClassifier=true`.
74+
- `tpStopAt`: optional, defaults to `9223372036854775807`
75+
76+
- Type: integer
77+
- Multivalued: False
78+
79+
80+
> Number of milliseconds into the video to stop processing, only applies when `useClassifier=true`.
81+
- `tpSampleRate`: optional, defaults to `1000`
82+
83+
- Type: integer
84+
- Multivalued: False
85+
86+
87+
> Milliseconds between sampled frames, only applies when `useClassifier=true`.
88+
- `useStitcher`: optional, defaults to `true`
89+
90+
- Type: boolean
91+
- Multivalued: False
92+
- Choices: `false`, **_`true`_**
93+
94+
95+
> Use the stitcher after classifying the TimePoints.
96+
- `tfMinTPScore`: optional, defaults to `0.5`
97+
98+
- Type: number
99+
- Multivalued: False
100+
101+
102+
> Minimum score for a TimePoint to be included in a TimeFrame. A lower value will include more TimePoints in the TimeFrame (increasing recall in exchange for precision). Only applies when `useStitcher=true`.
103+
- `tfMinTFScore`: optional, defaults to `0.9`
104+
105+
- Type: number
106+
- Multivalued: False
107+
108+
109+
> Minimum score for a TimeFrame. A lower value will include more TimeFrames in the output (increasing recall in exchange for precision). Only applies when `useStitcher=true`
110+
- `tfMinTFDuration`: optional, defaults to `5000`
111+
112+
- Type: integer
113+
- Multivalued: False
114+
115+
116+
> Minimum duration of a TimeFrame in milliseconds, only applies when `useStitcher=true`.
117+
- `tfAllowOverlap`: optional, defaults to `false`
118+
119+
- Type: boolean
120+
- Multivalued: False
121+
- Choices: **_`false`_**, `true`
122+
123+
124+
> Allow overlapping time frames, only applies when `useStitcher=true`
125+
- `tfDynamicSceneLabels`: optional, defaults to `['credit', 'credits']`
126+
127+
- Type: string
128+
- Multivalued: True
129+
130+
131+
> Labels that are considered dynamic scenes. For dynamic scenes, TimeFrame annotations contains multiple representative points to follow any changes in the scene. Only applies when `useStitcher=true`
132+
- `tfLabelMap`: optional, defaults to `[]`
133+
134+
- Type: map
135+
- Multivalued: True
136+
137+
138+
> (See also `tfLabelMapPreset`, set `tfLabelMapPreset=nopreset` to make sure that a preset does not override `tfLabelMap` when using this) Mapping of a label in the input TimePoint annotations to a new label of the stitched TimeFrame annotations. Must be formatted as IN_LABEL:OUT_LABEL (with a colon). To pass multiple mappings, use this parameter multiple times. When two+ TP labels are mapped to a TF label, it essentially works as a "binning" operation. If no mapping is used, all the input labels are passed-through, meaning no change in both TP & TF labelsets. However, when at least one label is mapped, all the other "unset" labels are mapped to the negative label (`-`) and if `-` does not exist in the TF labelset, it is added automatically. Only applies when `useStitcher=true`.
139+
- `tfLabelMapPreset`: optional, defaults to `relaxed`
140+
141+
- Type: string
142+
- Multivalued: False
143+
- Choices: `noprebin`, `nomap`, `strict`, `simpler`, `simple`, **_`relaxed`_**, `binary-bars`, `binary-slate`, `binary-chyron-strict`, `binary-chyron-relaxed`, `binary-credits`, `collapse-close`, `collapse-close-reduce-difficulty`, `collapse-close-bin-lower-thirds`, `ignore-difficulties`, `nopreset`
144+
145+
146+
> (See also `tfLabelMap`) Preset alias of a label mapping. If not `nopreset`, this parameter will override the `tfLabelMap` parameter. Available presets are:<br/>- `noprebin`: []<br/>- `nomap`: []<br/>- `strict`: ['`B`:`Bars`', '`S`:`Slate`', '`IN`:`Chyron-person`', '`CR`:`Credits`', '`M`:`Main`', '`O`:`Opening`', '`W`:`Opening`', '`Y`:`Chyron-other`', '`KU`:`Chyron-other`', '`L`:`Other-text`', '`G`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`T`:`Other-text`', '`P`:`-`', '`-`:`-`']<br/>- `simpler`: ['`B`:`Bars`', '`S`:`Slate`', '`IN`:`Chyron`', '`CR`:`Credits`', '`P`:`Neg`', '`-`:`Neg`']<br/>- `simple`: ['`B`:`Bars`', '`S`:`Slate`', '`IN`:`Chyron-person`', '`CR`:`Credits`', '`Y`:`Other-text`', '`KU`:`Other-text`', '`M`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`GLOTW`:`Other-text`', '`P`:`Neg`', '`-`:`Neg`']<br/>- `relaxed`: ['`B`:`Bars`', '`S`:`Slate`', '`Y`:`Chyron`', '`KU`:`Chyron`', '`IN`:`Chyron`', '`CR`:`Credits`', '`M`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`GLOTW`:`Other-text`', '`P`:`Neg`', '`-`:`Neg`']<br/>- `binary-bars`: ['`B`:`Bars`']<br/>- `binary-slate`: ['`S`:`Slate`']<br/>- `binary-chyron-strict`: ['`IN`:`Chyron-person`']<br/>- `binary-chyron-relaxed`: ['`Y`:`Chyron`', '`KU`:`Chyron`', '`IN`:`Chyron`']<br/>- `binary-credits`: ['`CR`:`Credits`']<br/><br/> Only applies when `useStitcher=true`.
147+
- `pretty`: optional, defaults to `false`
148+
149+
- Type: boolean
150+
- Multivalued: False
151+
- Choices: **_`false`_**, `true`
152+
153+
154+
> The JSON body of the HTTP response will be re-formatted with 2-space indentation
155+
- `runningTime`: optional, defaults to `true`
156+
157+
- Type: boolean
158+
- Multivalued: False
159+
- Choices: `false`, **_`true`_**
160+
161+
162+
> The running time of the app will be recorded in the view metadata
163+
- `hwFetch`: optional, defaults to `false`
164+
165+
- Type: boolean
166+
- Multivalued: False
167+
- Choices: **_`false`_**, `true`
168+
169+
170+
> The hardware information (architecture, GPU and vRAM) will be recorded in the view metadata
171+
172+
173+
#### Outputs
174+
(**Note**: "*" as a property value means that the property is required but can be any value.)
175+
176+
(**Note**: Not all output annotations are always generated.)
177+
178+
- [http://mmif.clams.ai/vocabulary/TimeFrame/v6](http://mmif.clams.ai/vocabulary/TimeFrame/v6)
179+
- _timeUnit_ = "milliseconds"
180+
181+
- [http://mmif.clams.ai/vocabulary/TimePoint/v5](http://mmif.clams.ai/vocabulary/TimePoint/v5)
182+
- _timeUnit_ = "milliseconds"
183+
- _labelset_ = a list of ["B", "CR", "E", "F", "GLOTW", "IN", "KU", "M", "P", "S", "Y"]
184+

‎docs/_apps/swt-detection/v8.4/metadata.json‎

Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
{
2+
"name": "Scenes-with-text Detection",
3+
"description": "Detects scenes with text, like slates, chyrons and credits. This app can run in three modes, depending on `useClassifier`, `useStitcher` parameters. When `useClassifier=True`, it runs in the \"TimePoint mode\" and generates TimePoint annotations. When `useStitcher=True`, it runs in the \"TimeFrame mode\" and generates TimeFrame annotations based on existing TimePoint annotations -- if no TimePoint is found, it produces an error. By default, it runs in the 'both' mode and first generates TimePoint annotations and then TimeFrame annotations on them.",
4+
"app_version": "v8.4",
5+
"mmif_version": "1.1.0",
6+
"app_license": "Apache 2.0",
7+
"identifier": "http://apps.clams.ai/swt-detection/v8.4",
8+
"url": "https://github.com/clamsproject/app-swt-detection",
9+
"input": [
10+
{
11+
"@type": "http://mmif.clams.ai/vocabulary/VideoDocument/v1",
12+
"required": true
13+
}
14+
],
15+
"output": [
16+
{
17+
"@type": "http://mmif.clams.ai/vocabulary/TimeFrame/v6",
18+
"properties": {
19+
"timeUnit": "milliseconds"
20+
}
21+
},
22+
{
23+
"@type": "http://mmif.clams.ai/vocabulary/TimePoint/v5",
24+
"properties": {
25+
"timeUnit": "milliseconds",
26+
"labelset": [
27+
"B",
28+
"CR",
29+
"E",
30+
"F",
31+
"GLOTW",
32+
"IN",
33+
"KU",
34+
"M",
35+
"P",
36+
"S",
37+
"Y"
38+
]
39+
}
40+
}
41+
],
42+
"parameters": [
43+
{
44+
"name": "useClassifier",
45+
"description": "Use the image classifier model to generate TimePoint annotations.",
46+
"type": "boolean",
47+
"default": true,
48+
"multivalued": false
49+
},
50+
{
51+
"name": "tpModelName",
52+
"description": "Model name to use for classification, only applies when `useClassifier=true`.",
53+
"type": "string",
54+
"choices": [
55+
"convnextv2_large",
56+
"convnextv2_tiny"
57+
],
58+
"default": "convnextv2_tiny",
59+
"multivalued": false
60+
},
61+
{
62+
"name": "tpModelBatchSize",
63+
"description": "Number of images to process in a batch for classification. Smaller batch sizes will use less memory but may be slower. The default value of 200 is set to be the safely maximum size for \"large\" model running on desktop-grade GPU (12GB VRAM). Only applies when `useClassifier=true`.",
64+
"type": "integer",
65+
"default": 200,
66+
"multivalued": false
67+
},
68+
{
69+
"name": "tpUsePosModel",
70+
"description": "Use the model trained with positional features, only applies when `useClassifier=true`.",
71+
"type": "boolean",
72+
"default": true,
73+
"multivalued": false
74+
},
75+
{
76+
"name": "tpStartAt",
77+
"description": "Number of milliseconds into the video to start processing, only applies when `useClassifier=true`.",
78+
"type": "integer",
79+
"default": 0,
80+
"multivalued": false
81+
},
82+
{
83+
"name": "tpStopAt",
84+
"description": "Number of milliseconds into the video to stop processing, only applies when `useClassifier=true`.",
85+
"type": "integer",
86+
"default": 9223372036854775807,
87+
"multivalued": false
88+
},
89+
{
90+
"name": "tpSampleRate",
91+
"description": "Milliseconds between sampled frames, only applies when `useClassifier=true`.",
92+
"type": "integer",
93+
"default": 1000,
94+
"multivalued": false
95+
},
96+
{
97+
"name": "useStitcher",
98+
"description": "Use the stitcher after classifying the TimePoints.",
99+
"type": "boolean",
100+
"default": true,
101+
"multivalued": false
102+
},
103+
{
104+
"name": "tfMinTPScore",
105+
"description": "Minimum score for a TimePoint to be included in a TimeFrame. A lower value will include more TimePoints in the TimeFrame (increasing recall in exchange for precision). Only applies when `useStitcher=true`.",
106+
"type": "number",
107+
"default": 0.5,
108+
"multivalued": false
109+
},
110+
{
111+
"name": "tfMinTFScore",
112+
"description": "Minimum score for a TimeFrame. A lower value will include more TimeFrames in the output (increasing recall in exchange for precision). Only applies when `useStitcher=true`",
113+
"type": "number",
114+
"default": 0.9,
115+
"multivalued": false
116+
},
117+
{
118+
"name": "tfMinTFDuration",
119+
"description": "Minimum duration of a TimeFrame in milliseconds, only applies when `useStitcher=true`.",
120+
"type": "integer",
121+
"default": 5000,
122+
"multivalued": false
123+
},
124+
{
125+
"name": "tfAllowOverlap",
126+
"description": "Allow overlapping time frames, only applies when `useStitcher=true`",
127+
"type": "boolean",
128+
"default": false,
129+
"multivalued": false
130+
},
131+
{
132+
"name": "tfDynamicSceneLabels",
133+
"description": "Labels that are considered dynamic scenes. For dynamic scenes, TimeFrame annotations contains multiple representative points to follow any changes in the scene. Only applies when `useStitcher=true`",
134+
"type": "string",
135+
"default": [
136+
"credit",
137+
"credits"
138+
],
139+
"multivalued": true
140+
},
141+
{
142+
"name": "tfLabelMap",
143+
"description": "(See also `tfLabelMapPreset`, set `tfLabelMapPreset=nopreset` to make sure that a preset does not override `tfLabelMap` when using this) Mapping of a label in the input TimePoint annotations to a new label of the stitched TimeFrame annotations. Must be formatted as IN_LABEL:OUT_LABEL (with a colon). To pass multiple mappings, use this parameter multiple times. When two+ TP labels are mapped to a TF label, it essentially works as a \"binning\" operation. If no mapping is used, all the input labels are passed-through, meaning no change in both TP & TF labelsets. However, when at least one label is mapped, all the other \"unset\" labels are mapped to the negative label (`-`) and if `-` does not exist in the TF labelset, it is added automatically. Only applies when `useStitcher=true`.",
144+
"type": "map",
145+
"default": [],
146+
"multivalued": true
147+
},
148+
{
149+
"name": "tfLabelMapPreset",
150+
"description": "(See also `tfLabelMap`) Preset alias of a label mapping. If not `nopreset`, this parameter will override the `tfLabelMap` parameter. Available presets are:\n- `noprebin`: []\n- `nomap`: []\n- `strict`: ['`B`:`Bars`', '`S`:`Slate`', '`IN`:`Chyron-person`', '`CR`:`Credits`', '`M`:`Main`', '`O`:`Opening`', '`W`:`Opening`', '`Y`:`Chyron-other`', '`KU`:`Chyron-other`', '`L`:`Other-text`', '`G`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`T`:`Other-text`', '`P`:`-`', '`-`:`-`']\n- `simpler`: ['`B`:`Bars`', '`S`:`Slate`', '`IN`:`Chyron`', '`CR`:`Credits`', '`P`:`Neg`', '`-`:`Neg`']\n- `simple`: ['`B`:`Bars`', '`S`:`Slate`', '`IN`:`Chyron-person`', '`CR`:`Credits`', '`Y`:`Other-text`', '`KU`:`Other-text`', '`M`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`GLOTW`:`Other-text`', '`P`:`Neg`', '`-`:`Neg`']\n- `relaxed`: ['`B`:`Bars`', '`S`:`Slate`', '`Y`:`Chyron`', '`KU`:`Chyron`', '`IN`:`Chyron`', '`CR`:`Credits`', '`M`:`Other-text`', '`F`:`Other-text`', '`E`:`Other-text`', '`GLOTW`:`Other-text`', '`P`:`Neg`', '`-`:`Neg`']\n- `binary-bars`: ['`B`:`Bars`']\n- `binary-slate`: ['`S`:`Slate`']\n- `binary-chyron-strict`: ['`IN`:`Chyron-person`']\n- `binary-chyron-relaxed`: ['`Y`:`Chyron`', '`KU`:`Chyron`', '`IN`:`Chyron`']\n- `binary-credits`: ['`CR`:`Credits`']\n\n Only applies when `useStitcher=true`.",
151+
"type": "string",
152+
"choices": [
153+
"noprebin",
154+
"nomap",
155+
"strict",
156+
"simpler",
157+
"simple",
158+
"relaxed",
159+
"binary-bars",
160+
"binary-slate",
161+
"binary-chyron-strict",
162+
"binary-chyron-relaxed",
163+
"binary-credits",
164+
"collapse-close",
165+
"collapse-close-reduce-difficulty",
166+
"collapse-close-bin-lower-thirds",
167+
"ignore-difficulties",
168+
"nopreset"
169+
],
170+
"default": "relaxed",
171+
"multivalued": false
172+
},
173+
{
174+
"name": "pretty",
175+
"description": "The JSON body of the HTTP response will be re-formatted with 2-space indentation",
176+
"type": "boolean",
177+
"default": false,
178+
"multivalued": false
179+
},
180+
{
181+
"name": "runningTime",
182+
"description": "The running time of the app will be recorded in the view metadata",
183+
"type": "boolean",
184+
"default": true,
185+
"multivalued": false
186+
},
187+
{
188+
"name": "hwFetch",
189+
"description": "The hardware information (architecture, GPU and vRAM) will be recorded in the view metadata",
190+
"type": "boolean",
191+
"default": false,
192+
"multivalued": false
193+
}
194+
],
195+
"est_gpu_mem_min": 400,
196+
"est_gpu_mem_typ": 2000
197+
}

‎docs/_apps/swt-detection/v8.4/submission.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"time": "2025-12-14T01:08:09+00:00",
3+
"submitter": "keighrim",
4+
"image": "ghcr.io/clamsproject/app-swt-detection:v8.4"
5+
}

0 commit comments

Comments
 (0)