For the charity concert Berlin-Reykjavik-Damascus we needed to connect and remix live streaming from three countries. After some research, considering the no budget situation and the international audience for this occasion, we decided that Facebook live was the way to go. On the other hand, I wanted to have an easy and intuitive way to switch all the different sources involved: Power Point Presentation, USB-Webcam, HDMI Videocam, Excel Sheet with endless Line-Up, two Facebook live streams, Mobile Phone Camera, a Premiere Pro Timeline, Generative TouchDesigner Szenes (*.tox), Feedback and Screen Capture.
Further more I wanted to be able to use the flexibility of TouchDesigner's native interface to mess with the incoming signals in all kinds of experimental ways. I found that NewTeks NDI format is a great solution to get all kinds of sources in and out of TouchDesigner. It was like a flashback to 1999, when I participated in the worldhausTV project and we had 99 days to experiment with a semi-professional TV studio. Lots of cameras, mixers and sources. It was crazy times and ever since I have been fallen for live video.
The bad news? NDI is only available in the commercial version of TouchDesigner099 and while you can send audio through NDI, TouchDesigner doesn't seem to support that.
However, it is not crucial to have the commercial version for this Tutorial, nor is audio. Indeed you can achieve most of what I have described with OBS Studio, an open source streaming platform, that we are also going to use to encode our TouchDesigner Output to Facebook live. What you can not achieve with OBS though
is all the beautiful mindfuck, that you can do live with TouchDesigner, while you are broadcasting.
However, it is not crucial to have the commercial version for this Tutorial, nor is audio. Indeed you can achieve most of what I have described with OBS Studio, an open source streaming platform, that we are also going to use to encode our TouchDesigner Output to Facebook live. What you can not achieve with OBS though
is all the beautiful mindfuck, that you can do live with TouchDesigner, while you are broadcasting.
As these are two different things, it doesnt matter if you have the commercial version or not, so just come along and follow me through this tutorial! To get you started, here are some things to donwload and install on your computer. I am writing this from a PC/WIN10 perspective. Most of it works on a Mac, except the NDI Plugin for OBS, which is the most comfortable, but not the only way to get your video output into OBS.
So, let's get started!
So, let's get started!
Download and Install
Palakis' NDI Plugin for OBS (Win only)
MAKE NDI WORK
Now, that we have installed all this, lets start by making us familiar with the NDI Tools. NDI allows you to share Video Sources in a network. So you should be in a network when you try this right now. The better the network, the more crazy you can get. First, open Test Patterns
You are now sending your first NDI source into the network, a test pattern.
You can open more of these windows and send more test patterns if you want. Next, open Studio Monitor
You can open more of these windows and send more test patterns if you want. Next, open Studio Monitor
In the upper left corner you find the menue. Here you can select all the NDI sources in the network.
It does not matter if they are open on the same computer or on another network computer. Again you can open more than one Studio Monitor to preview different sources. Next, open Scan Converter
It does not matter if they are open on the same computer or on another network computer. Again you can open more than one Studio Monitor to preview different sources. Next, open Scan Converter
Nothing seems to happen, but there is a new icon in the notification area of your taskbar and
when you right-click that icon, you can select which windows you want to capture as NDI sources.
It even works to have two or more Chrome windows open in fullscreen and get the respective outputs as different NDI sources (while having TouchDesigner open in the foreground). By the way, with the Adobe CC NDI Plugins, you can send NDI streams directly from the timeline of PremierePro and Character Animator.
when you right-click that icon, you can select which windows you want to capture as NDI sources.
It even works to have two or more Chrome windows open in fullscreen and get the respective outputs as different NDI sources (while having TouchDesigner open in the foreground). By the way, with the Adobe CC NDI Plugins, you can send NDI streams directly from the timeline of PremierePro and Character Animator.
And now, comes the moment we all have been waiting for! Open TouchDesigner.099
Delete the default scene and close the palette browser. Press TAB to open the operator menue and lay down
NDI in TOP, NDI out TOP, Movie file in TOP Nodes. In the NDI in parameters notice that you can select all the NDI sources on the network, our Test Pattern among them. Select the Test Pattern and give it a little time, before the signal becomes available. Connect an info CHOP to see the resolution of the source.
NDI in TOP, NDI out TOP, Movie file in TOP Nodes. In the NDI in parameters notice that you can select all the NDI sources on the network, our Test Pattern among them. Select the Test Pattern and give it a little time, before the signal becomes available. Connect an info CHOP to see the resolution of the source.
Now, connect the Movie file in TOP to the NDI out TOP,
give the NDI stream a name in the parameter window. Use another info CHOP to learn about this nodes resolution. We will work with 1280x720 in this tutorial, just to make it easier on everybody. Go back to the NDI Studio Monitor and select the source coming from TouchDesigner. Voila, your TD output is another NDI source.
give the NDI stream a name in the parameter window. Use another info CHOP to learn about this nodes resolution. We will work with 1280x720 in this tutorial, just to make it easier on everybody. Go back to the NDI Studio Monitor and select the source coming from TouchDesigner. Voila, your TD output is another NDI source.
CONNECT TOUCHDESIGNER WITH OBS
First, open Open Broadcast Studio. Set it to Studio Mode (lower right corner) and create a scene (lower left corner). A scene can hold different sources. You can add new sources to the scene by pressing the plus button.
Select NDI Source give it a name and press ok in the pop-up window. In the next window select the TouchDesigner NDI stream and press OK. The signal from TD is now a source in your OBS scene.
You can place and scale it inside the window. When you are happy with a scene you can send it to the output by pressing "Transition" in the middle between the Screens. You can build many scenes with countless sources and easily organise your broadcast like that. When you don't have a commercial TD license you can still grab your TouchDesigner output via screen capture and send it to facebook live. However, the NDI way is very comfortable and offers a much wider range of creative possibilities.
CONNECT OBS STUDIO WITH FACEBOOK LIVE
This is a bit different if you start facebook live from a personal account or a page. I am going to show you how to do it for a page. While you can feed facebook live directly via screen capture the results I experienced were very bad and OBS made everything much better. Open your facebook page and above the cover image select "Content Options". In the list on the left, go to Videos/Video collection. In the Top Bar of the list with the videos is a "+LIVE" button. Press it to create a Facebook live stream.
In the following window, you can give the stream a name, tag it etc. Most importandly though, you can copy the Stream Key here.
Go to OBS, open Settings (lower right corner) and go to Stream (second from the top). In the Platform menue select Facebook live and paste the Stream Key where it says Stream Key.
Now, you can fiddle with all of the settings for hours if you want, but I guess you can also just leave everything as is and enjoy your stream rather sooner than later. To do that, press Start Stream (lower left corner) and send the stream to facebook.
Shortly after you should be able to see the signal on facebook. Now you only need to press the blue button in the lower right corner to go live. While you are live, there is a red LIVE sign in the top left corner of the stream. In the lower right corne is a button to stop the stream. Check out, how the stream looks in your timeline. Now, you can keep on streaming while you proceed with the rest of the tutorial or not, your pipeline is established : )
BUILDING THE SWITCHER : INPUT
We are going back to TouchDesigner and start with a clean slate.
Then we lay down a container COMP with a resolution of 1280x720, rename it to Switcher and dive into it.
Now we lay down an In TOP and make 16 copies of it. I originally designer the switcher to work with a MPC style 16 pad MIDI controller, but my implementation has still problems, so for this tutorial we are going to stick to 16 sources, but we are build a GUI to operate it. It is more intuitive anyway as you can directly select the image. From these 16 inputs, we want to be able to select an A and a B channel, of which one is going live to the output, while we can select the next source offline on the other one.
Thus we lay down two Switch TOPs, rename them to switch_A and switch_B and connect the 16 In TOPs to each of them. An elegant way to do that is to select all the In TOPs and then connect the first one of them to the target switch. We have created the beautiful centerpiece of our network and because like ourselves some style, we lay down everything nicely around the center axis of TD.
Now we lay down a Cross TOP and connect the switches to it. From there we wire on to a Out TOP.
Lets zoom out and have a look at our container: 16 Ins and one Out. Next we need to create some placeholder content and the possibility to name our sources. To do that we create another container COMP, source_0 by name and dive into it.
Lay down a Text TOP with Output Resolution 1280x720, change Font to Arial and Size to 400, set Auto-Size Font to Auto-Fit if Too Large.
Set Font Color to orange and type this expressions
into the Background Color parameters:
into the Background Color parameters:
me.parent().digits/32
me.par.bgcolorr
me.par.bgcolorr
me.par.bgcolorr
me.par.bgcolorr
and in the Text field write:
me.parent().digits
I will explain later what the effect of this expression is. Get a glimpse by unfolding that parameter and pressing the light green button, so this becomes interpreted as an expression. Do the same for the Background Color expressions if they havent been turned on automatically yet.
Connect an In Top, a Fit TOP and another Text TOP to that. Connect an Out TOP to the Fit TOP whose Output Resolution is again 1280x720 and who is set to Fit Vertical and makes sure we have no incompatible resolutions inside the Switcher.
In the Text parameter of the second Text TOP write the following expression and turn it on.
str(me.parent().par.Title)
Hmm. This one gives us an error. Its all red and unfriendly. Lets have a closer look:
The error message basically says: There is no Parameter called Title in the object, that is my parent.
And it's true, there isn't and that's why we are going to put it there now. The parent in this case is the container COMP we have been working in. So lets move up one level and give this "parent" a parameter called Title. To do so, right click onto the container and select Customize Component.
The error message basically says: There is no Parameter called Title in the object, that is my parent.
And it's true, there isn't and that's why we are going to put it there now. The parent in this case is the container COMP we have been working in. So lets move up one level and give this "parent" a parameter called Title. To do so, right click onto the container and select Customize Component.
Up pops the Component Editor. We add a page called source and add a parameter to that page. Choose str for string. That means we will be able to type text into it later. Call the parameter Title.
Note that there is a new page in the parameter window of our container and we can type into it something like: Camera 1
Lets dive back into the container and make some adjustments.
First set the Font Size of text2 to 80 and Auto-Size to No Auto-Fit. Set Horizontal Align to Left and Vertical Align to Top. Set Border Space to 20 and change Font to Arial.
Then go to text1 TOP and on the Color page change Background Alpha to 1.
We could have done that before, but we forgot.
We could have done that before, but we forgot.
Move one level up and and the Common Page of the source_0 container set the Node View to Operator Viewing and the Operator Viewer parameter to ./text2
Lay down a Replicator COMP. Set Replication Method to By Number and set the Number of Replicants to 16. Make sourceOperator Prefix source_ and Master Operator source_0.
When you zoom out, you see that there are 16 new nodes now, spread out in a grid and not in the best position.
Change Layout to Vertical and move the nodes to a good place by changing the Layout Origin parameters.
Before we have written different expressions into parameters. These were using me.parent().digits
This way, we use the automatically created names of the replicants to drive the content inside the node, like the number and the Background Color.
This way, we use the automatically created names of the replicants to drive the content inside the node, like the number and the Background Color.
Later you can feed all your NDI and TD signals into these nodes. For now we have nice looking generative placeholders for our further proceedings. Connect the outlets of the source_[1-16] replicants with the In[1-16] inlets of the Switcher container.
BUILDING THE SWITCHER : SELECTION
Jump back into Switcher and lay down a container COMP called Selector_A.
Set Output Resolution to 640 x 360, set Node View to Default Viewer and give it ./out_A as Background TOP.
Jump inside and create a typical render setup: Camera COMP, Light COMP, Geometry COMP and Render TOP.
Jump into the geo1 COMP, delete the default torus and create a Rectangle SOP.
Set sizex = 1 and sizey = me.par.sizex * 9 / 16 - Remember to turn Display and Render Flags on.
Set sizex = 1 and sizey = me.par.sizex * 9 / 16 - Remember to turn Display and Render Flags on.
Set Render TOP to 640x360. Add a Constant MAT and give it to the geo COMPs Material parameter.
Connect the Render TOP to an Out TOP. Rename it to Out_A.
Connect the Render TOP to an Out TOP. Rename it to Out_A.
Lay down a GridSOP, sizex = 4 , sizey = me.par.sizex*9/16 with 4 Rows and 4 Columns.
Connect it to a Sopto CHOP, a Merge CHOP and a Null CHOP.
Connect it to a Sopto CHOP, a Merge CHOP and a Null CHOP.
Rename the Null CHOP to Instancer. Add a Pattern CHOP, Type Ramp with Length 16, Amplitude of 2, set Channel Name to tex and set the To Range to 1 to 16 (16 discreet values).
If you make the node viewer active , right click and select Dot per Sample, you get a better feel for what we are doing.
If you make the node viewer active , right click and select Dot per Sample, you get a better feel for what we are doing.
Now wire the pattern output into the Merge CHOP. We have used the Grid SOP to define 16 distinct points in space. We added a rising number to each of these points through the Pattern CHOP. Now we can use these channels to place objects onto that coordinates. Go to the geo1 node and turn Instancing on. Drag the Null CHOP called Instancer onto the Instance CHOP/DAT field. Now you can use the channels tx, ty and tz for the TX, TY and TZ parameters.
On the Instance 2 page you can use tex for the Texture Index Parameter. In the Instance Textures field write:
/project1/source_[1-16]/text[1-16]
This means, that for each of the 16 instances there is a texture with the number tex in name
on the top level of the project, inside the source_ [1-16] containers. If your render still doesnt show anything, move the light and the camera to a higher z position. In my case 6.1 for the camera and 5 for the light work well.
on the top level of the project, inside the source_ [1-16] containers. If your render still doesnt show anything, move the light and the camera to a higher z position. In my case 6.1 for the camera and 5 for the light work well.
Adjust the size of grid1 SOP and rectangle1 SOP (lives inside geo1 COMP) to spread the instances more even.
For me sizex = 3.75 for grid1 SOP and sizex = 1.2 for rectangle1 look allright. The respective sizey is automaticall kept in a 16/9 aspect through the expressions.
For me sizex = 3.75 for grid1 SOP and sizex = 1.2 for rectangle1 look allright. The respective sizey is automaticall kept in a 16/9 aspect through the expressions.
Next we want to make the preview tiles clickable so that we can select sources that way. Back into Selector_A we lay down a Panel CHOP and in the Select field we write u v select. That way we learn the position and select state of the mouse that navigates the container in which the Panel CHOP lives.
Feed that to a Chopto DAT (turn Include Names on) and to a Renderpick DAT. Set Strategy to Hold First Picked, Pick Radius to 3, Render TOP to render1 and on the Options page set Fetch Position to In Camera Space and turn on Fetch Instance ID.
Continue with a Select DAT, Set Select Rows to by Index, Start and End Row Index to 1 and set Select Cols to By Name. Set Start and End Col Name to Instance. That way we only get one number: The ID of the tile that has been clicked.
With a Datto CHOP we fetch that data into the CHOP world, set Output to Channel per Column and First Row is to values. Next we send this into a Rename CHOP and rename it to sourcea. Next we add a Null CHOP, set the Cook Type to Selective and turn Check Values, Check Names and Check Range on. That means this Null will only cook when there is actually some change in the values. This makes our patch faster.
Now we lay down an Out CHOP and feed sourcea into it. We will use it later. We also run it into a Fan CHOP with the Channel Names set to chan[1-16] and Outside Range set to Set Channels to 0.
From here we go into a Shuffle CHOP, Method set to Swap Channels and Samples.
Now add a Filter CHOP and turn Filter per Sample on. Add again a selective Null CHOP, to contain cooking. From here split the signal up through two Math CHOPs The first has the Range 1 to 0.75. We will use this to scale down the selected instance. The second one gets a Range from 0.75 to 1.5. We will use this one to highlight the selected tile. Rename the first channel to select and the second to hilite.
Feed them into the Merge CHOP. Now the new values are available to be used on the instances.
On the Instance page of the geo1 node, set the Parameters SX and SY to select.
On the Instance page of the geo1 node, set the Parameters SX and SY to select.
On the Instance 2 page, set the A parameter to hilite.
If you zoom out of the container and turn it active, you can now select preview tiles by clicking them. Connect Null operators to the containers outlets to see what is coming out of them.
Lets take sourcea and use it to drive switch_A and turn Blend between Inputs on.
Now the switch should output the source that you clicked on.
Now the switch should output the source that you clicked on.
Copy Selector_A and the Null coming from it, paste and rename to Selector_B. Jump inside, navigate to rename1 CHOP and change it to sourceb. Use sourceb to drive switch_B.
BUILDING THE SWITCHER : SWITCHING
Now that we have the ability to select 2 sources out of 16, we want to be able to switch between them. As we can select the sources by clicking on them, we should use the same method to select the output. We add another Container COMP by the name of Select_A, 640x360 by size and jump into it. Here we connect a Panel CHOP to an Out CHOP. Set the Panel CHOPs Select Parameter to select only. We want to know if this container has been clicked or not. We do not care where it has been clicked this time.
Further we add an In TOP, a Rectangle TOP and an Out TOP. For the Rectangle TOP set Resolution to 1280x720. Also set Size units to Pixel and Size to 1280x720, Fill Alpha to 0, Border Color to orange and
Border Width to 10 Pixels.
Border Width to 10 Pixels.
We will use this orange frame later to highlite the source that is currently on Air. Right now we do not know yet, so we have should create on central place that knows which state our switcher is in. Move on level up and create a Constant CHOP called state.
Plug the output of switch_A into Select_A and set Select_As Background TOP parameter to ./out1
Add a Null CHOP and a Chop Execute DAT that monitors that Null. Off to On only.
Use this expression in the Chop Execute DAT:
op('state').par.value0 = 0
Whenever Select_A gets selected, state becomes zero.
Lets duplicate that behaviour for Select_B, but so that state becomes one when Select_B is selected.
Now, when you activate the Select_A/B containers and click them,
you should see state jump between zero and one.
Now, when you activate the Select_A/B containers and click them,
you should see state jump between zero and one.
To use this to drive our crossfade, we add a Lag CHOP and a selective Null CHOP which we call x_fade.
Use this value to drive the cross1 TOP.
Now we want to implement the frame that shows us which source is currently on Air.
Let's jump into the Select_A container and write the following expression into the Border Alpha parameter:
Let's jump into the Select_A container and write the following expression into the Border Alpha parameter:
1-op('../state')['chan1']
The frame will be visible when state is zero, which means, Select_A is live.
We repeat this for Select_B, but this time with this expression:
We repeat this for Select_B, but this time with this expression:
op('../state')['chan1']
To finish up the interface we want to layout the different elements of the switcher. To that end we get an emptypanel component from the Palette Browser (UI/TUIK/Windowparts) and size it 1280x360.
On the Children page we set Align to Left to Right.
On the Children page we set Align to Left to Right.
Then we connect Selector_A and Select_A into the emptypanel. The Selector should be on the left side. If they appear in the wrong order, go to the second container and on the first page set parameter Align Order to 1.
This should make the containers switch order.
This should make the containers switch order.
Repeat this Step for Selector_B and Select_B.
Move up one level and for the Switcher container, on the Children page set Align to Top to Bottom.
BUILDING THE SWITCHER : OUTPUT
Next to the Switcher container create another container COMP, set the Size to 1280x720 and call it Output.
Jump inside and connect an In TOP with a NDI out TOP.
Jump back out and connect Switcher to Output. On the Look page of Output set Background TOP to ./in1
I am assuming, that you have two monitors, one for the switcher, one for the output. You could lay down two Window COMPs, feed them the containers and assign them to your screens.
Maybe a bit more effective would be to collect all outputs in one container and open up just one Window COMP. In that case I would arrange the outputs vertical and get a very reasonable sized
128 x1440 resolution vs 2560 x 720 in a horizontal arrangement.
128 x1440 resolution vs 2560 x 720 in a horizontal arrangement.
Obviously you would have to setup your output screens accordingly.
Personally I would want to avoid Performance Mode with this setup because I want to have full access to the Network Editor, to play with my sources. The final output could either be previewed on a third screen or just streams out into the internets...
The Switcher is ready to be filled with all kinds of sources. Obviously you do not have to stick to NDI Streams but can use everything that TouchDesigner can read or create.
I am not claiming that any of my programming is done the most effective way.
I am still learning and would love to hear your feedback about this method.
I am still learning and would love to hear your feedback about this method.
Happy Switching!
You can download the project file here:
https://www.dropbox.com/s/1vf5onkqxd7enge/MX.NDI.SWITCHER.TD.toe
https://www.dropbox.com/s/1vf5onkqxd7enge/MX.NDI.SWITCHER.TD.toe
