AS3.0: boosting .flv volume by hacking the NetStream object out of FLVPlayback

Use Case
You’re handed an AS3.0 project that’s been built by someone else and that developer used the FLVPlayback component to play multiple videos. All’s worked OK in the past, but suddenly, you get 1 video, already in .FLV format and it’s volume is way lower than all the rest of the videos. The client’s going to notice, everybody knows this much. When you ask the logical question: why not just send the file back to the video editor and have her boost the volume in After Effects or Audacity? You’re told no, it’s too late, the video budget’s been spent already and nobody caught the problem during video assets QA.

Lucky for them, you can use AS3.0 to boost the volume of a video with low sound quality.

This is easier to do with custom video players because by default, the FLVPlayback component doesn’t give you any sort of getNetStream() method. Thanks to a good Developer at Firefly there’s a way to grab the NetStream from FLVPlaback. Once we have it, you combine it with a SoundTransform instance and you’re good to go.

package com.yourpackage.path
{
	//...
	import flash.media.SoundTransform;	
	import flash.net.NetStream;
    import flash.display.Sprite; 
    import fl.video.FLVPlaback; 

	public class MyPlayer extends Sprite 
	{
		private var videoSound:SoundTransform = new SoundTransform();
		private var ns:NetStream;
        private var vidPlayer:FLVPlaback = new FLVPlaback();
		//...
	
		public MyPlayer()
		{
			//...
			//used for NetStream capture
			idPlayer.addEventListener(VideoEvent.STATE_CHANGE, onVideoStateChange);
			//...
		
			boostVolume();
		}
	
		// grab the NetStream using the evt.vp index
		public function onVideoStateChange(e:VideoEvent)
		{
			ns = vidPlayer.getVideoPlayer(e.vp).netStream;
		}
	
		public function boostVolume():void
		{			
			videoSound.volume = 2.4;
			ns.soundTransform = videoSound;
		}
	
		public function normalizeSound():void
		{
			videoSound.volume = 1;
			ns.soundTransform = videoSound;
		}
		
	    //...
	} 
}

Some examples of the State design pattern

If you’ve ever been sick of writing convoluted if/else statements, this design pattern is a good thing to look into.

One thing it enables you to do in Actionscript — have the same button behave differently, depending on what State the application is in. For example, check out your average Flash video player, like this YouTube video from Crossfit Chesapeake / Wilkes Weightlifting:
http://crossfitchesapeake.com/2009/09/13/wod-mon-91309.aspx

In it’s embedded form, the video has a giant PLAY button with a hit area that covers the width & height of the video. In this example, this button could have three states:

  • State 1:
    If the video is dormant (hasn’t been played since the page loaded), clicking the button plays the video. The play icon graphic is displayed in the dormant State disappears once the button is clicked.
  • State 2:
    If the video is already playing, i.e. the application is in a different State, the same button behaves differently. At this time the play button click handler takes you to the YouTube.com page that originally hosts this video and pauses the embedded video.
  • State 3:
    If you go back to the embedded version of the video which has been paused by the play button’s click handler method (the new window with opened with youtube.com, etc) and click the video (not the navbar at the bottom), the smae “click handler” function from the above functionality resumes video play.

And of course, when the same “click handler” method behaves differently based on the State of the application, it’s an example of polymorphism, one of the 4 main elements of OOP. Here’s a decent definition:

Polymorphism allows two objects to be treated identically, using the same methods, even though the objects implement these methods in quite different ways. It is this concept of “same appearance, different behavior” that gets the 0 word, polymorphism.

There’s another State pattern example in the AS3 port of an open source physics library called Box2DFlashAS3.

The Main.as file, in this case the first file that gets the application going, has a listener that calls a function called update(). Among other things, this function
a) continuously checks for an id variable (altered by the user pressing a keyboard key) and
b) sets a variable of type Test to one of the subclasses of Test, like TestRagdoll (the default example when you run the SWF).

The state machine code is essentially this part of Main.as:

/* ...	*/
switch(m_currId){
	// Ragdoll
	case 0:
		m_currTest = new TestRagdoll();
		break;
	// Compound Shapes
	case 1:
		m_currTest = new TestCompound();
		break;
	// Crank/Gears/Pulley
	case 2:
		m_currTest = new TestCrankGearsPulley();
		break;
	// Bridge
	case 3:
		m_currTest = new TestBridge();
		break;
	// Stack
	case 4:
		m_currTest = new TestStack();
		break;
	// CCD
	case 5:
		m_currTest = new TestCCD();
		break;
	// Theo Jansen
	case 6:
		m_currTest = new TestTheoJansen();
		break;
		
	/* ...	*/
}
/* ...	*/

Testing a cool AS3-based video player

Martin Legris created a great little Actionscript-only AS3 Video player:

today I will explain how to play .FLV files that are hosted on a standard HTTP server, no fancy streaming. You can use pre-made components to do so, but sometimes, for whatever reason, you want to do it yourself. I wrote my first FLV playing algorithm in AS3 about 10 months ago, it has evolved since and here is the breakdown on how I make it work. It’s been used in many widgets, mostly for Music Nation.

His tutorial came in three parts, with final code located here, in Part 3.

Here’s a quick little Client class I wrote as an Actionscript Project in Flex Builder 3. If you’re using the Flash IDE, just use this as the Document Class. If you’re using FlashDevelop — you know what to do. This code can be used to test the code in Part 3:

package {

	import ca.newcommerce.media.FLVPlayer;
	import ca.newcommerce.media.MediaData;

	import flash.display.Sprite;
	import flash.events.MouseEvent;
	import flash.media.Video;
	import flash.text.TextField;
	import flash.text.TextFieldAutoSize;

	public class FLVPlayerTest extends Sprite
	{
		private var myData:MediaData;
		private var vid:Video;
		private var myPlayr:FLVPlayer;
		private var txtBox:TextField;
		private var txtBox2:TextField;

		//in real life, populate this array from an external data source
		private var trackArray:Array = ["http://www.helpexamples.com/flash/video/clouds.flv",
             "http://www.helpexamples.com/flash/video/typing_short.flv",
             "http://www.helpexamples.com/flash/video/sheep.flv",
             "http://www.helpexamples.com/flash/video/water.flv",										       "http://www.helpexamples.com/flash/video/cuepoints.flv"];

		public function FLVPlayerTest()
		{
			inti();
			addBasicMenu();
		}

		public function inti():void
		{
			//start the player
			myData = new MediaData(trackArray[0], "some title", 300, "image.jpg", 320, 200);
			myPlayr = new FLVPlayer();
			vid = new Video(320, 200);

			myPlayr.video = vid;
			myPlayr.playMedia(myData);

			stage.addChild(vid);
		}

		public function addBasicMenu():void
		{
			//add some simple text boxes to test loading other videos
			txtBox2 = new TextField();
         	txtBox2.y = vid.height + 20;
         	txtBox2.autoSize = TextFieldAutoSize.LEFT;
         	txtBox = new TextField();
         	txtBox.autoSize = TextFieldAutoSize.LEFT;
         	txtBox.multiline = true;
         	txtBox.x = vid.x;
         	txtBox.y = txtBox2.y + 20;
         	txtBox.mouseEnabled = true;
         	stage.addChild(txtBox);
         	stage.addChild(txtBox2);

         	txtBox2.text = "Click a line below to load another movie:\n\n";

         	txtBox.appendText("line one\n");
         	txtBox.appendText("line two\n");
         	txtBox.appendText("line three\n");
         	txtBox.appendText("line four\n");
         	txtBox.appendText("line five\n");

			txtBox.addEventListener(MouseEvent.CLICK, handleClick, false, 0, true);
		}

		public function handleClick(e:MouseEvent):void
		{
			//tells us what line of the text box was clicked by the mouse
			var lineIndx:int = txtBox.getLineIndexAtPoint(e.target.mouseX, e.target.mouseY);

			trace(lineIndx);
			switch(lineIndx)
			{
				case 0:
					changeVid(trackArray[lineIndx]);
					break;
				case 1:
					changeVid(trackArray[lineIndx]);
					break;
				case 2:
					changeVid(trackArray[lineIndx]);
					break;
				case 3:
					changeVid(trackArray[lineIndx]);
					break;
				case 4:
					changeVid(trackArray[lineIndx]);
					break;
			}
		}

		public function changeVid(v:String, t:String="", l:int=120, img:String="", w:int=320, h:int=200):void
		{
			//calls the get uri() function of the MediaData class,
			//since myData is an instance of (or object of type) MediaData
			myData.uri = v;
			myData.title = t; // get title();
			myData.duration = l; // get duration();
			myData.width = w;
			myData.height = h;
			myPlayr.play(false);

		}
	}
}

I guess FLVPlayerTest(), as it’s written above, can be considered a View and a Controller in an MVC pattern. It can be a Controller, because of the handleClick() & changeVid() functions that tell the model to update itself. addBasicMenu() could be part of the Controller, since it ads a visual element whose sole purpose is to receive user input (mouse click events) and then tell the Model to update itself. Then again… it can also be part of the View, since it’s a visual component that the user sees and code associated with it can be considered to be display logic. In the MVC chapter of Advanced ActionScript 3 with Design Patterns by Joey Lott and Danny Patterson, put similar visual elements into their Controller. In their Clock example, the Controller class includes code for the button that toggles between Views and text fields that tell the Model to update itself based on user input.

If I add buttons to control play, pause, volume, those would be part of the Controller as well, if we stick with the example from Joey Lott and Danny Patterson.

Although the FLVPlayer class feels like a Controller, it’s actually the main part of the Model. If this was a middleware application, it would be the DB class, or the class that has all bulk of the app’s business logic. It would contain the CRUD methods that interact with the Database (in this case the video data source).

MediaData is part of the Model too. It encapsulates the properties of a video, like uri, image, title, duration, etc. The trackArray is part of how the View stores info from the Model, or at least, it would be, if it was coming from an external source. In real life it would be an Array of MediaData objects. I’m thinking of how in Budi Kurniawan‘s simple servlet-based MVC application a Database class (Model) grabs info from the database (external source) and instantiates objects (part of Model) that store the DB data for use by the application. In the same servlet app, the View then runs a foreach loop and populates a collection object like a List, HashMap or Vector with all the objects instantiated by the DB class.

Adding some error checking

If there’s 1 thing I learned from OOAD folks, it’s this: avoid writing “The Kitchen Sink” class. In other words, make sure your classes don’t try to do everything at once. Some purists demand that there be 1 task per class. Others are OK with 2 – 3 tasks per class. In this AS3 video example, I separated the Error Checking task and the Meta Data processing task into their own separate classes. This helped reduce size & length of the Main class and parceled the Error Checking and Meta Data functionality out for easier re-use in other Flash / Flex video projects.  

Main.as, the client / driver / document class:

package
{
	import utls.ErrorChecker;
	import vid.CustomClient;
	import flash.net.NetStream;
	import flash.net.NetConnection;
	import flash.events.NetStatusEvent;
	import flash.events.IOErrorEvent;
	import flash.events.AsyncErrorEvent;
	import flash.events.SecurityErrorEvent;
	import flash.display.Sprite;
	import flash.media.Video;

	[SWF (width="600", height="400", backgroundColor="#9999cc", frameRate="30")]
    public class Main extends Sprite
	{
		private var ec:ErrorChecker;
		private var nc:NetConnection;
		private var ns:NetStream;
		private var theUrl: String = "http://www.helpexamples.com/flash/video/cuepoints.flv";

		public function Main()
		{
			nc = new NetConnection();
			nc.connect(null);

			ns = new NetStream(nc);
			ns.bufferTime = 10;
			ns.client = new CustomClient();
			ns.receiveAudio(true);
			ns.receiveVideo(true);
			ns.play(theUrl);

			ec = new ErrorChecker();

			// check for errors on the NetConnection
			nc.addEventListener(NetStatusEvent.NET_STATUS, ec.doNetStatus);
			nc.addEventListener(IOErrorEvent.IO_ERROR, ec.doIOError);
			nc.addEventListener(SecurityErrorEvent.SECURITY_ERROR, ec.doSecurityError);

			// check for errors on the NetStream
			ns.addEventListener(AsyncErrorEvent.ASYNC_ERROR, ec.doAsyncError);
			ns.addEventListener(NetStatusEvent.NET_STATUS, ec.doNetStatus);
			ns.addEventListener(IOErrorEvent.IO_ERROR, ec.doIOError);

			var vid:Video = new Video();
			vid.attachNetStream(ns);
			addChild(vid);
		}
	}
}

the ErrorChecker Class:

package utls
{
	import flash.events.NetStatusEvent;
	import flash.events.AsyncErrorEvent;
	import flash.events.IOErrorEvent;
	import flash.events.SecurityErrorEvent;

    public class ErrorChecker
	{
		public function ErrorChecker() { }

		public function doSecurityError(e:SecurityErrorEvent):void
		{
			// crossdomain xml file should fix this error
                        trace("AbstractStream.securityError:"+e.text);
		}

		public function doIOError(e:IOErrorEvent):void
		{
			trace("AbstractScreem.ioError:"+e.text);
		}

		public function doAsyncError(e:AsyncErrorEvent)
		{
			trace("AsyncError:"+e.text);
		}

		public function doNetStatus(e:NetStatusEvent):void
		{
			trace(e.info.code);
		}

    }
}

the CustomClient class:

package vid
{
	import flash.events.NetStatusEvent;
	import flash.events.AsyncErrorEvent;
	import flash.events.IOErrorEvent;
	import flash.events.SecurityErrorEvent;

    public class CustomClient
	{

		private var cuePointCount:int = 0;

        public function onMetaData(infoObject:Object):void
		{
            trace("metadata");
			trace("duration =" + infoObject.duration);
			trace("width =" + infoObject.width);
			trace("height =" + infoObject.height);
			trace("framerate =" + infoObject.framerate + "\n");
        }

		public function onCuePoint(infoObject:Object):void
		{
			cuePointCount += 1;
			switch(cuePointCount)
			{ //the flv has 3 cue points created @ encoding time
				case 1:
					trace("cue point VONE!");
					cuePntDetails(infoObject);
					break;
				case 2:
					trace("cue point TWO!");
					cuePntDetails(infoObject);
					break;
				case 3:
					trace("cue point THREEE!");
					cuePntDetails(infoObject);
					break;
			}
		}

                //can function as an onComplete only for streaming, not progressive download
		private function onPlayStatus(infoObject:Object):void
		{ //only works with Flash Media Server / streaming video, not progressive download
			switch (infoObject.info.code)
			{
				case "NetStream.Play.Complete":
					vidComplete();
					break;
			}
			trace(infoObject.info.code);
		}

		public function cuePntDetails(infoObject):void
		{
			trace("name: " + infoObject.name);
			trace("time: " + infoObject.time);
			trace("type: " + infoObject.type); //navigation or event
			trace("params: " + infoObject.parameters + "\n");
		}

		public function vidComplete():void
		{
			trace("NetStream.Play.Complete = Video complete");
		}

    }
}

Playing around with Metadata & Cue Points

Adobe has a helpful intro to Cue Points & Metadata in Flash video.

I started off with this part of the tutorial:

Extending the NetStream class and adding methods to handle the callback methods

//...
var ns:NetStream = new NetStream(nc);
ns.client = new CustomClient2();
//...

and expanded the CustomClient class to see if I can squeeze more information out of the .flv file:

package 
{
    public class CustomClient2 
	{
		
		private var cuePointCount:int = 0;
				
        public function onMetaData(infoObject:Object):void 
		{
            trace("metadata");
			trace("duration =" + infoObject.duration);
			trace("width =" + infoObject.width);
			trace("height =" + infoObject.height);
			trace("framerate =" + infoObject.framerate + "\n");
        }

		public function onCuePoint(infoObject:Object):void 
		{
			cuePointCount += 1;
			switch(cuePointCount)
			{ //the flv has 3 cue points created @ encoding time
				case 1:
					trace("cue point VONE!");			
					cuePntDetails(infoObject);
					break;
				case 2:
					trace("cue point TWO!");							
					cuePntDetails(infoObject);
					break;
				case 3:
					trace("cue point THREEE!");							
					cuePntDetails(infoObject);
					break;
			}
		}
		
		public function cuePntDetails(infoObject):void 
		{
			trace("name: " + infoObject.name);
			trace("time: " + infoObject.time);
			trace("type: " + infoObject.type); //navigation or event
			trace("params: " + infoObject.parameters + "\n");
		}
		
		private function onPlayStatus(infoObject:Object):void 
		{ //only works with Flash Media Server / streaming video, not progressive download? 
			switch (infoObject.info.code) 
			{
				case "NetStream.Play.Complete":
					vidComplete();
					break;
			}
			trace(infoObject.info.code);
		}
		
		public function vidComplete():void  
		{
			trace("NetStream.Play.Complete = Video complete");	
		}		
    }
}