---
title: "Improve Media #3"
video_id: H-xmOFVWlrM
url: https://www.youtube.com/watch?v=H-xmOFVWlrM
workflow_page: https://workflows-of-ai.com/workflow/improve-media-3
language: en
captions: generated
---

0:01 hey everyone Marcus here um so you see
0:03 YouTube in front of me yesterday I was
0:05 talking about uh the media module how to
0:08 improve and add metadata to like stuff
0:10 like you know images and documents and
0:11 stuff and those things are fairly easy
0:14 to do what I said yesterday is also as
0:17 soon as there is a service that does
0:19 video we're going to add that but that
0:21 got me thinking that I mean there there
0:24 there is actually a way to solve this
0:25 today so uh I started developing that
0:28 yesterday night just because I got
0:31 inspired and now there is a rule that
0:34 can basically describe a video so that's
0:37 what I want to show you now um I have
0:40 just you know ripped a video from
0:42 YouTube so this is the video that was
0:46 talking about uh that I'm going to use
0:48 as example just so you can you can
0:51 basically rerun this yourself to make
0:54 sure uh that this works but also see the
0:57 description that I create now actually
1:00 has something to do with what you see
1:01 and what you hear in this video right uh
1:04 I will explain how this works but but
1:07 because the um generation takes a long
1:10 time I will just start setting up
1:12 everything so uh the AI interpolator is
1:15 already installed AI inator open AI is
1:17 already installed with a key I just uh
1:20 enabled the media module because that's
1:22 what we're going to be working with
1:25 here uh then we go to Media types and
1:28 Video in this case then
1:31 uh and we have a video file so I can
1:32 just you know show you what you get
1:35 basically if we do a video here so I
1:39 just going to upload this video
1:41 here uh save it and then we have a video
1:45 file right uh and I will I'm not allowed
1:48 to climb this Tower I did ask but not
1:50 only just climbing this yeah you see
1:52 that is that video right without AIO
1:55 anything um what we're going to do now
1:58 is that we want want some metadata from
2:01 this right and in this case I'm just
2:03 going to do uh the simplest thing which
2:06 is basically describe it so I have only
2:09 added this so far to text formatted long
2:13 in theory this could be added like the
2:16 image description to any type of field
2:18 you know is there emails in this field
2:20 uh does this video or sorry is there
2:24 emails in this video represented or are
2:26 they talking about emails or uh so it
2:29 becomes a May in that case or if it's a
2:32 Boolean
2:34 like it's this video about Tom Scott
2:37 which it is in this case and get checked
2:39 and so on so I could add that uh just
2:42 comment or create a ticket if you think
2:44 that's actually something that would be
2:45 useful it will take some time to do this
2:48 but um I could do that if if people find
2:51 it
2:52 useful the reason I'm a little bit
2:54 hesitant is because I think open AI will
2:57 you know come with a feature for this
3:01 too I hope that's better than what I
3:04 have done here but that's if you want
3:05 something in the meantime basically so
3:08 I'm just going to do a video description
3:10 here field so it's formatted text takes
3:12 formatted long that's the only place
3:13 this rule works at the
3:15 moment uh and as I said this is only in
3:18 the dev Branch you would have to get the
3:19 dev Branch version of it uh it's not
3:22 stable in this case and then I just
3:25 enable the AI interpolator and there is
3:27 this rule open AI video
3:31 text uh and we just say which base field
3:35 we do and then basically what we want to
3:38 get out of it so describe the the video
3:43 from start to
3:47 finish in one value in up to five
3:53 paragraphs with up to four sentences
3:58 each you make may use HTML tags like
4:05 headers
4:07 uh strong
4:10 italic
4:11 underline oh it's not italic it's
4:15 em
4:17 um also describe the video graphy so you
4:22 can see because otherwise it would be
4:24 easy to just get it from the
4:25 transcription what's actually happening
4:27 right um
4:30 and how it matters to your description
4:35 and once again this is prompting um you
4:38 have to write the right prompt and you
4:40 have to write what you want out the
4:41 prompt also so you could ask
4:43 basically uh do we do we see Tom Scott
4:46 in this video and it would probably give
4:48 you a very short answer like yes Tom
4:50 Scott is in this video If it knows about
4:52 Tom Scott which I assume it because uh
4:55 what's important to know here there's no
4:57 Advanced uh prompting in this case you
5:00 can take many tokens I could fix this
5:02 later if you can think of a reason to do
5:05 this but right now there's actually not
5:07 even placeholders so you just write what
5:09 you want
5:10 basically uh normally you should
5:13 definitely do bat share uh sorry cron
5:17 because this is going to take forever
5:19 but I have I have long timeout so I can
5:22 run it on the web server locally no
5:23 problem I just do that it here you can
5:26 also say that you only want the video to
5:30 matter or only the audio but in this
5:32 case we're going to do both I just save
5:35 this uh and then I go back to the
5:38 content uh ah sorry some
5:42 media and we're going to have that field
5:45 under here yeah we should probably move
5:48 it up by the way so it looks a little
5:52 bit better oh sorry I was supposed to
5:54 press manage form display there video
5:57 description will put there on the video
5:59 file save reload so you can see that the
6:02 video description is empty right we
6:04 click save and this is going to take
6:05 forever so while we're doing this I'm
6:08 going to show you what is happening now
6:10 so it's of course it's transcribing the
6:13 video but then it's also uh taking and
6:17 this is not my idea I'm not I mean I
6:20 have thought about it but this guy on
6:24 like open AI Forum gave me a much better
6:26 idea so my idea was basically take each
6:29 of the uh scene Cuts so this is you oh
6:33 by the way this is using FFM peg in the
6:34 background you need FFM Peg to be able
6:36 to do this if you don't have FM Peg the
6:39 rule is not going to show up uh but
6:42 anyway with FFM Peg you can do something
6:44 that basically says that if if there's a
6:46 scam cut which is usually where key
6:49 frames happen so if you know video you
6:50 know what a key frame is but if there's
6:52 a scene
6:53 cut uh take a screenshot if something
6:56 completely different happens take a
6:58 screenshot and my idea was to send all
7:00 these images but then I realized you
7:02 know that was a this Tom Scot video is
7:04 like a 7 minute
7:07 video uh it would generate like 60
7:10 images so then this guy said like okay
7:13 we can take a video and his case he just
7:16 taking every second or something I think
7:18 and generate uh um like a rer from it
7:23 so basically what I did was to combine
7:26 these two things where I said that oh
7:29 it's uh finished by the way just so you
7:32 know how long it takes but before we see
7:33 the results I'm just going to explain
7:35 everything
7:37 uh that we restor all these key frames
7:40 in multiple images and send all these
7:42 images and I also add the time stamp so
7:45 it can basically connect that whatever
7:49 is said at 6 seconds that comes back
7:52 from the transcription as have being
7:54 said at 6 seconds is connected to this
7:56 image right uh uh and that's it so we're
8:01 sending all these restor images we're
8:03 sending the transcript and basically
8:04 tell us tell open AI this is a video
8:07 please answer the
8:08 prompt uh and then you get something
8:11 back like this oh here we have the Json
8:13 error again so this would basically be
8:15 something like this I
8:18 guess uh so the video is the history and
8:20 current St mod blah so uh yeah you could
8:25 just you know you can just P pause the
8:28 video on YouTube and compare this to the
8:30 video that uh Tom Scott made because you
8:33 know which video it is uh and and then
8:38 you can basically see that it's using
8:42 kind of the video you can see here by
8:44 the way because this you could get from
8:46 transcription but the footage feature
8:48 various shots that include aial views
8:50 closeup of towers historical references
8:53 so you can see that it also actually
8:55 takes the
8:56 images uh so while this is not a uh
8:59 definitely not a perfect solution of
9:02 doing video
9:03 descriptions compared to the models that
9:06 will come in I'm sure not that long time
9:11 there is a way to do it today so yeah
9:17 that's it uh if you want to try it you
9:19 have to get the dev version as I Said So
9:22 yeah thank you for watching