---
title: "Video"
video_id: tW_vJQhKzC4
url: https://www.youtube.com/watch?v=tW_vJQhKzC4
workflow_page: https://workflows-of-ai.com/workflow/video
language: en
captions: generated
---

0:01 hey there so Marcus here H and for once
0:05 I'm on camera as well and that actually
0:07 has a reason because we're going to use
0:09 me as context so it might be important
0:11 that you at least have seen me uh the
0:14 context we're going to use though is a
0:16 image of me so this image where I wear
0:18 glasses think about that as well when
0:19 you see the end result uh and we take
0:22 this web page as well which is the page
0:25 from the AI interpolators documentation
0:28 that is about the the AI
0:31 interpolator uh and we're just going to
0:33 take this link and we're going to take
0:35 this image and out from the AI inter
0:38 player we're going to have a video of a
0:39 avatar of me talking in my voice in
0:43 rhyme about the history of the AI interl
0:46 and this is completely dynamically
0:48 created so you can take any link and
0:51 take any image and get the Avatar of
0:53 that image talking in any kind of 11
0:56 Labs voice uh about anywh website so
1:01 while this is pretty gimmicky to be
1:03 honest the did module can be used for a
1:07 lot of less gimmicky stuff say for
1:09 instance that you have a workflow that
1:13 where you have a user that is highend
1:16 for you that has registered on your
1:17 website you could then have the CEO of
1:20 your company welcoming that person in a
1:22 personalized video which is pretty cool
1:25 right or you could actually do instead
1:29 of a chat box but if someone ask a
1:31 question you could actually answer with
1:33 a video answering the question so
1:35 there's a lot of possibilities there um
1:39 as usual this is open source this is
1:42 free so this is based on rupo which is a
1:45 Content management system this is the AI
1:47 interpolator that I built on top of this
1:51 uh so I will show the workflow end to
1:53 end how to set it up it requires no code
1:55 it's just basically clicking around to
1:58 set it up uh but before we do that you
2:02 can be a judge for yourself if you want
2:05 to actually continue with this so I will
2:07 show the end result
2:10 first in the realm of tech where ideas
2:13 fuse and sprawl the AI interpolator
2:16 arose Standing Tall from merging of text
2:19 it got its Birthright name now with 50
2:21 plus rules it's no longer the same it
2:23 aims to create workflows chains that
2:26 bind to generate content leaving none
2:29 behind with future dreams of Agents
2:32 project leads in jira and slack
2:34 fulfilling all your needs
2:37 finished so before I just uh start
2:41 showcasing how to set this up I just
2:43 want to remind people that haven't seen
2:45 it there's a website called workflows of
2:48 AI uh so it has dashes in it where you
2:52 can basically look at all the workflows
2:54 I have done so far uh to see what you
2:58 can do with AI later so yeah look into
3:02 this web page if you don't know it about
3:04 it you can of course follow me subscribe
3:07 on YouTube If you rather do this but on
3:09 this web page you can filter by which
3:11 mod modules it has been used and so on
3:14 so uh yeah look into that workflows of
3:18 AI so um what we're going to use as
3:22 context to start with is this page it's
3:24 part of the uh documentation for AI
3:29 Interpol
3:31 so uh it's if you just Google drup
3:35 interpolator documentation or something
3:37 you're going to find it there's a sub
3:38 page here that's talking about the
3:40 history of why this was
3:43 built
3:46 um on top of that then the plugins can
3:49 just show you here on available plugins
3:51 we're going to use the ID to generate
3:53 the video we're going to use 11 Labs
3:55 field to generate the
3:57 audio uh we're going to use open AI
4:00 to generate a new image so it's going to
4:03 describe my image and then it's going to
4:04 generate a new image of this uh and
4:08 we're going to also use the simple
4:12 crawler to basically crawl this page
4:16 then
4:18 so this is druple
4:21 um standard installation I just
4:24 installed these plugins then I have set
4:26 them up so if you go to web service D
4:28 config for instance you have to add your
4:30 D ID uh key same with LM lab settings
4:34 and so on and if you're not from the
4:36 droup world just Google installing
4:38 modules in grou this is the image by the
4:42 way I'm going to use as context so
4:44 that's an image of me it's going to
4:46 basically describe this and generate
4:48 another image of this is the
4:50 idea um so we're start by adding a
4:54 content type that we call Avatar video
5:00 just disable some stuff here that's
5:01 typical CMS
5:03 stuff uh so if you're not from the
5:06 druple
5:07 world this is the way you can do it in
5:10 the gooy but if you want to create
5:12 something scaling you should use
5:13 something called custom entities remove
5:16 the body field we don't need it in this
5:18 case we're not going to create a web
5:19 page and then we start filling in the
5:22 fields first of all the fields that are
5:24 required to fill in so that's going to
5:26 be the link so the link where it fetches
5:29 the col text we're going to use external
5:31 links only we're going to disable link
5:33 text save that uh sorry we're also going
5:37 to I edit it again we're going to make
5:39 it required because that's one of the
5:40 fields we need to fill
5:42 in create another field that is required
5:46 and that's the image field uh so this is
5:49 IM no Source image we call that and we
5:55 save uh we make this also required we we
5:59 don't need a alt field it's not used for
6:02 our web page and then we're going to
6:04 start interpolating so we start with the
6:06 link to HTML so that's formatted text
6:09 text format long we call that
6:12 HTML so where it's going to take the
6:14 link now it's going to scrape it and put
6:16 it into this HTML field right so we say
6:19 full HTML we allow all HTML we're going
6:22 to use the rule simple crawler it's
6:25 going to be in base mode based on the
6:27 link field so it takes the link fields
6:29 scrapes it and puts it into this field
6:32 uh we add a user agent here I
6:36 think. requires a use agent uh we do
6:40 article segmentation so that basically
6:41 makes sure that we're just getting the
6:44 actual article and not all the links
6:46 here on the right side or the header and
6:48 so on and we set a very low weight I'm
6:51 going to use batch mode in this case if
6:54 you're doing this for real especially
6:55 with the video you should do it on the
6:57 CR job because it takes time to generate
6:59 videos or in a q
7:02 worker so that's our first step oh let
7:06 me check something I'm going to set a
7:08 really low weight if I didn't do that as
7:10 well no I did 50 because this has to run
7:13 first right from this now we want to
7:17 generate an audio so I'm just going to
7:21 do first of all I'm going to call this
7:24 the real MP3 so that's the real audio
7:27 that we're going to store uh uh this is
7:31 because of how the 11 Labs Field Works
7:34 I'm going to try to create 11 Labs field
7:37 to did rule so you don't have to take
7:40 this uh step in between but for now I
7:44 will do this basically you have to allow
7:46 MP3 on this so this is going to take the
7:48 generat dialogue uh and put this in a
7:51 MP3 right uh all right so this has a bug
7:56 here this is not going to happen when
7:57 you do it I need to fix a B here on the
8:00 did
8:01 module uh and then we're going to create
8:04 the 11 La Field and that's going to be
8:06 in charge of generating a dialogue from
8:09 the body of the HTML right so this we
8:13 call
8:16 dialog and since this split up in many
8:19 we have to set unlimited and that's why
8:21 we want to concatenate in this case
8:23 right uh so we're going to use the real
8:26 MP3 and we're going to con concatenate
8:29 using F fmeg enable the AI interpolator
8:32 we're going to do the dialogue Creator
8:34 so that creates dialogue in 11 Labs
8:37 we're going to base it on the HTML and
8:40 then I'm going to write a prompt here
8:42 with the voice and here we need to use
8:45 placeholders just search Marcus so I'm
8:48 going to use my voice that I've already
8:50 trained the levels on with the voice
8:52 Mark is your hands on create a four
8:57 sentence rhing
9:02 uh
9:04 monologue
9:07 on the history
9:11 of the the thing written in the
9:22 context uh and here we use the context
9:25 and context in this case is the HTML so
9:28 basically we're saying with this voice
9:29 create a four sentence rhyming
9:32 monologue about the history uh based on
9:35 the HTML
9:37 right this has to be higher of course
9:40 than the HTML we say bet share as well
9:43 we use GPT for uh turbo because we need
9:47 good results GPT 3.5 just doesn't do
9:50 good enough results in this case
9:52 especially if we want
9:53 rhyming uh and then we have the audio
9:57 generated so then the next step comes
10:00 that we want to do a image
10:03 description right so we take plain text
10:07 TT plane
10:09 long and this is then going to be enable
10:13 in inator we're going to base this on
10:16 the image so we do string long we do
10:19 Advance mode in this case oh so I have
10:21 installed the token module that's
10:23 something that you need to install to
10:24 get the advanced mode up and
10:26 running uh because in this case we're
10:29 not basing it on some other field text
10:32 field we're basing it on an actual image
10:34 field right so we just write a prompt
10:37 here based on the
10:39 image WR
10:43 um
10:44 write one
10:47 paragraph
10:49 of
10:55 uh
10:56 for C actually I'm just going to do this
11:00 I'm going to set this unlimited because
11:02 sometimes it might generate a
11:04 really it might split it up because when
11:07 I'm asking for four sentences it might
11:09 read this that it wants to create four
11:11 different values right so because of
11:14 that I said unlimited so we don't just
11:16 get one of the pieces so write one
11:18 paragraph of four
11:21 sentences uh with a
11:24 descriptive
11:27 text descripted text
11:33 describe being the person in the image
11:37 in
11:39 detail mention
11:42 things like color of CR what type of
11:48 CRS
11:50 gender
11:56 um
11:58 calculate a
12:00 age
12:02 um
12:04 yeah hair color skin type
12:10 glasses or other
12:14 peral no per I don't know the word other
12:19 uh other
12:22 accessories
12:24 um yeah that's that's enough
12:28 also
12:31 describe the background
12:39 shortly this is for an image
12:44 prompt also add short uh depth of field
12:50 in the end so that's what I have on my
12:53 image maybe it forgets it so I just adds
12:56 it there so he um
12:59 or it remembers to generate this and
13:02 context then in this case no sorry I
13:06 don't need a context I'm stupid we don't
13:08 have a context it's just the image and
13:11 under advanced settings now we set lower
13:13 weight batch it's important that you use
13:16 gp4 Vision preview because that that is
13:19 the only thing that can read images and
13:20 here you can then set a image field to
13:23 use and we use the SCE
13:25 image
13:27 so this this will generate
13:31 this then I will generate a file upload
13:35 which is a image and this will be the
13:38 created image then so this is what we're
13:40 going to uh create the image on based on
13:45 that prompt that we had so once again I
13:49 disable alt field we don't need it it's
13:51 not a web page enable a inator dolly
13:54 we're going to generate images we base
13:56 it on the image description
14:02 context
14:04 um
14:06 and that's it we just take the image
14:09 description as it is right set this to
14:12 90 batch
14:15 dolly3 uh we want HD and a neutral
14:22 image and this will create the image and
14:24 the last thing we need to create then is
14:27 another file upload which is the video
14:31 Avatar which is a file that will be of
14:35 type
14:37 MP4 enable AI interpolator we use the ID
14:40 avatar from audio base mode we're going
14:43 to use the real MP3 as the base field so
14:46 that's the audio we created and then the
14:48 expression of uh the person it generates
14:52 pH is
14:53 neutral uh I'm going to use image field
14:56 not the image URL and we're going to use
14:58 the created image in this case right and
15:01 we set the highest weight we set 110 for
15:05 just to be sure if I had 100
15:07 somewhere um and now it's going to chain
15:10 all these things
15:12 together on the manage display I will
15:15 just make sure that I just show the
15:17 video we don't care about the other
15:19 thing we just care about the result
15:22 right uh and this should be a video then
15:26 and we save that and then we're going to
15:28 create this so we create the Avatar
15:30 video I have to set a title because I'm
15:33 using content or notes content types in
15:36 trle so I will just set uh video here
15:39 and then link take this
15:43 link this is the correct right yeah why
15:46 was it build
15:47 future and uh then I'm going to take my
15:51 screenshot here that I have and that's
15:54 it the rest I'm not going to fill in
15:56 that's going to be filled in now so now
15:58 it's going to start changing hopefully
15:59 five rules yes because that's what we
16:02 added four out of these five rules will
16:05 be simple um to generate the video
16:11 generation takes time just so you know
16:14 that and that's why you need to put it
16:15 in a Chron job unless you have like me I
16:18 have on my local server that it just
16:21 doesn't time out the the the PHP worker
16:25 or the engine x uh server so
16:29 but that comes with complications or
16:32 security problems of course if you don't
16:35 know much about rupo but you thought
16:37 this was cool still um and if this was
16:41 too fast then please as I said before I
16:45 have this website called workflows of AI
16:48 where you can look at simpler workflows
16:51 to basically get the idea of what each
16:54 of these field interpretations I'm doing
16:57 does and how chaining works because I do
17:01 understand that the workflow I did now
17:03 was very
17:06 complex uh we will see if it works so I
17:10 I actually when I do these videos I
17:13 don't edit and that's why you have to
17:14 sit and wait now but that's good also it
17:17 gives you a realistic view of how long
17:19 time stuff takes to make
17:22 um but that also means that if if if
17:25 there's something going wrong I will
17:27 show that because
17:29 with AI it's not deterministic which
17:32 means that it can generate stuff that is
17:36 not actually what I thought at the first
17:38 iteration you might have to reprompt and
17:40 so on so because of that actually
17:44 it's yeah it is what it is now something
17:47 went wrong because the last rule should
17:49 not take so long
17:54 time yeah so uh there was a cut there by
17:57 the way I found the problem basically
17:59 the image that has been generated is so
18:01 big that it did won't accept it so I
18:05 have added into the did Rule now that it
18:07 will compress the image so you will not
18:10 run into this problem here you can by
18:13 the way see that it has generated a lot
18:15 of fields uh so it's just a video Avatar
18:18 that we need to do so I'm just
18:21 resaving and we will see if it can also
18:25 generate the video where I'm talking in
18:27 rme well the description of me is
18:30 talking and
18:32 rying
18:37 um so I just so you know I want as I
18:40 said I want to be honest when I'm
18:42 showcasing stuff because I know a lot of
18:46 startups are
18:47 showing things that seem completely you
18:51 know it's blowing your mind what it can
18:53 do with the AI but it's cherry picking
18:55 results in this case I'm trying to show
18:57 you what the AI inator actually does and
19:00 where you can run into
19:02 problems so let's look at this in the
19:05 realm of tech where ideas fuse and
19:07 sprawl the AI interpolator arose
19:09 Standing Tall from merging of texts it
19:12 got its Birthright name now with 50 plus
19:15 rules it's no longer the same it aims to
19:17 create workflows chains that bind to
19:20 generate content leaving none behind
19:23 with future dreams of Agents project
19:25 leads in jira and slack fulfilling all
19:28 your needs
19:31 finished right so we have something um
19:36 for The Voice by the way you can there
19:38 is 11 Labs voice field so you can
19:40 actually you could make that into a a
19:44 selection
19:46 here
19:48 uh actually let's try that as well I'm
19:51 just going to do this makes the video a
19:54 little bit longer but it might be
19:55 interesting for you uh so I'm just going
19:58 to create a new field here that we is
20:01 the 11 field voices and we're going to
20:03 choose
20:05 voice uh and let's see here uh we just
20:11 take
20:12 that so this is a dynamic field that
20:15 gets generated automatically right
20:20 uh I'm just going to make it required I
20:25 just realize we have to do to make sure
20:27 it's filled out
20:29 and the cool thing we can do then is
20:33 that in the
20:35 dialogue we can actually do copy this
20:40 but take advanced
20:42 mode so context is the HTML right so we
20:46 just take it in token
20:48 mode and we take here value so that is
20:53 filled in but then we're going to take
20:56 also the voice here which is is here
21:00 voice value so now this is going to be
21:04 dynamically filled in right the store is
21:06 still going to be the same so I uh will
21:11 just
21:14 do well there's one more thing I will do
21:16 here so I'm just going
21:18 to take the voice and put it up here
21:24 after Source
21:26 image so
21:29 and I have another image I've been
21:31 playing around with here that's huge
21:34 huge
21:36 Jackman huge Jackman not
21:40 huge
21:41 uh Jackman and we do angular history
21:48 maybe history of
21:51 angular
21:55 uh let's see here
21:59 something I mean this is really long but
22:01 that could be fun to see if it can take
22:03 such a long it should be able to uh and
22:07 then we have the image here of Mr
22:10 Jackman so it's this image where he's
22:12 sitting down and we have to Cho choose a
22:15 voice here so I'm going to choose uh uh
22:20 uh Scott Parker that's the voice I'm
22:22 using for my blog uh for my podcast by
22:25 the way so if you haven't seen this
22:27 there is podcast cast workflows of
22:31 ai.com where you can go and listen to a
22:34 a completely autogenerated podcast that
22:37 comes weekly about Drupal news so just
22:40 go to this podcast. workflows ai.com to
22:44 get that uh and I'm just going to save
22:47 now and this is the newest Caster's
22:49 voice in this case then and we will see
22:53 what happens so this will take some time
22:56 again I will just mute your myself I'm
22:59 not going to say anything interesting
23:01 you can just forward if you want in the
23:03 YouTube video to see the
23:27 results
23:57 e
24:27 e
24:32 right
24:34 so doesn't look that much like you get
24:37 one it looks like Lou Figo almost but
24:40 let's see from Angular JS in its prime a
24:42 transformation we've seen over time I'll
24:46 for to angular with its might making web
24:49 apps a delightful site through versions
24:52 and years it grew adapting as web
24:54 standards flew this journey of angular's
24:57 flight a testate to its developers
25:01 Insight right so that was just another
25:04 example of uh this getting
25:07 generated now this was very gimmicky of
25:10 course but what you can use this for
25:13 because you can see it it's still pretty
25:15 fast if you're a website for instance
25:19 that caters to uh high paying
25:23 individuals for instance you could have
25:26 a personalized video of this CEO of your
25:29 company welcoming them to the website
25:32 right so after they have registered
25:34 there's a video the next time they log
25:36 in where the CEO says hello Marcus
25:39 Johansson in my case Welcome to our
25:42 website blah blah blah it's personalized
25:44 right uh and there's a lot of other cool
25:47 stuff I think you can do with this um so
25:50 yeah thank you for
25:52 watching