Installing PhantomJS on NodeJS on Windows behind corporate proxy

Attempting to install phantomjs using npm, got an ETIMEOUT error, as you can see below

C:\Users\UserName\Downloads>npm install -g phantomjs
npm http GET https://registry.npmjs.org/phantomjs
npm http 304 https://registry.npmjs.org/phantomjs
npm http GET https://registry.npmjs.org/rimraf
npm http GET https://registry.npmjs.org/adm-zip/0.2.1
npm http 304 https://registry.npmjs.org/adm-zip/0.2.1
npm http 304 https://registry.npmjs.org/rimraf
npm http GET https://registry.npmjs.org/graceful-fs
npm http 304 https://registry.npmjs.org/graceful-fs
C:\Users\UserName\AppData\Roaming\npm\phantomjs -> C:\Users\UserName\AppData\Roaming\npm\node_modules\phantomjs\bin\phantomjs
 
> phantomjs@1.9.0-3 install C:\Users\UserName\AppData\Roaming\npm\node_modules\phantomjs
> node install.js
 
Requesting C:\Users\UserName\AppData\Roaming\npm\node_modules\phantomjs\tmp\phantomjs-1.9.0-windows.zip
 
events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: connect ETIMEDOUT
    at errnoException (net.js:878:11)
    at Object.afterConnect [as oncomplete] (net.js:869:19)
npm ERR! phantomjs@1.9.0-3 install: `node install.js`
npm ERR! `cmd "/c" "node install.js"` failed with 8
npm ERR!
npm ERR! Failed at the phantomjs@1.9.0-3 install script.
npm ERR! This is most likely a problem with the phantomjs package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node install.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls phantomjs
npm ERR! There is likely additional logging output above.
 
npm ERR! System Windows_NT 6.1.7601
npm ERR! command "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "-g" "phantomjs"
npm ERR! cwd C:\Users\UserName\Downloads
npm ERR! node -v v0.10.3
npm ERR! npm -v 1.2.17
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     C:\Users\UserName\Downloads\npm-debug.log
npm ERR! not ok code 0

I configured npm to use my corporate proxy, and every other package has downloaded and installed successfully, I'm guessing the phantomjs installer is doing something different. Rather than figure out what it's doing different I just downloaded phantomjs-1.9.0-windows.zip to the package so it could install without downloading.

Navigate to

C:\Users\UserName\AppData\Roaming\npm-cache\phantomjs\1.9.0-3\package

Created dir ‘tmp’

Downloaded phantomjs-1.9.0-windows.zip from https://code.google.com/p/phantomjs/downloads/list

Copy phantomjs-1.9.0-windows.zip into the tmp dir, located at

C:\Users\UserName\AppData\Roaming\npm-cache\phantomjs\1.9.0-3\package\tmp

Now I used 7-zip to re-create the package.

Navigate to

C:\Users\UserName\AppData\Roaming\npm-cache\phantomjs\1.9.0-3

TAR the package dir, so you'll create package.tar

Then gzip package.tar, naming it package.tgz, this will overwrite the current file.

Then run the installer again and it will use the local phantomjs-1.9.0-windows.zip to complete the install.

Hacking Proxy support into Brackets

Started using Brackets at work, though they use a corporate proxy, so I did a quick hack to get proxy support into Brackets for installing Extensions.

I did this for Brackets Sprint 24.

Open brackets/www/extensibility/node/ExtensionManagerDomain.js

Find the _cmdDownloadFile function, line 288
Add

proxy: ‘http://proxy.address:port’,

To the var req = request.get({});
For authenticated proxies,

proxy: ‘http://username:password@proxy.address:port’,

var req = request.get({ proxy: ‘http://proxy.address’, url: url, encoding: null },

Hope this helps, I'm guessing they will add proxy support down the line, but until then, this will get it working.

HowTo: jqspm - jQuery Selector Performance Monitor

Quick Start:

Head over to the jqspm repo, grab the files and checkout the README.

So I'll assume you've grabbed the files and have read the brief README. So here I'll cover how jqspm (jQuery Selector Performance Monitor) monitors selectors, and how to interpret the results.

How it works

When you call jqspm.start() the script wraps the existing jQuery() and jQuery.find() functions in monitoring functions. These monitoring functions capture what selectors are being passed in, they then call the original jQuery functions and record how long it took for them to execute, and how many elements they returned.

Once you have monitored some calls you'll want to printReport, this is where there may be some confusion, so I'll take some time explaining all the points I think might be confusing. Below is a picture of a report printed via console.table.

Selectors column
Here are the list of selectors that were called during the monitoring. You will notice they are separated by an =>, so the following:

$('UL').find('LI');

Will result in a selector looking like:

"UL => LI"

As you can see from the picture above, this will keep going on the longer you chain calls to jQuery() or jQuery.find().


totalCount column
This is the total number of elements that were returned by the whole chain of selectors.


Counts column
This is a list of the number of elements each part of the selector chain returned. So a selector like:

$('UL').find('LI');

Will return a count of:

[ 3, 11 ]

This tells you it found 3 ULs, and 11 LIs.

totalTime column
This is the combine time it took to find all elements that were returned by the chain of selectors. This is a good number to look at to pick up inefficient selector chains.

Timings column
This is a list of timings for each selector in the chain, so using the same chain as above:

$('UL').find('LI');

Will return:

[ 3, 5 ]

Oddities

Single Id selectors:
Queries like

$('#menuHolder').find('UL');

 Will only return counts/timings for the UL selector. I am looking at adding in the ability to monitor single ID selectors, but it's not high on the priority list as that query should only ever return 1 element, and should be instant.

The problem this currently posses is counts and timings for the above query will only contain 1 number, even though there are 2 selectors occurring, just keep that in mind when investigating results.

Chaining saved selectors:
Queries like

var $obj=$('BODY');
$('UL');
$obj.find('UL');

Will give you 3 results

"BODY"
"UL"
"BODY => UL"

The one I want to bring your attention to is the last one, it's counts and timings will look like this

[ "", 3 ]

This is because you chained a saved selector, $obj, so you know that BODY was from a previous selector.

What to look for

Here are a few screenshots showing how this can help, this is a real selector example I've found in production code. Though in the production environment there were a ton of other elements within each LI, this was causing the iPhone 5 to take 8.5 seconds to execute. By changing it all into the one selector that time was cut to about 50ms.

When you see a large number of elements returned early in the query chain, you should look at ways to optimise. In the IE8 example below, jQuery found 1557 LIs, and each of these was added to an array of elements.

It then loops through that array, executing the second selector (UL) on each element, so it executes it 1557 times. Each time it does this it needs to copy the elements it found back to the main collection array, and in browsers with poor memory management, or systems with limited memory, this can be costly.

Notice how when we moved the second selector into the first query, eliminating the need to loop, query, add to array, we get much better performance.

IE8

Nexus 4 - Chrome

Wrap it up

Hope that has made sense, and you find this useful, let me know any questions you have, or feedback, I'll update this page once I add more features.

jQuery Selector Performance Monitor - jqspm

In my job as a contract front end developer, I work with large existing code bases. Many times I've started at a company and been asked, "Why does Page X take so long to load?".

There are plenty of tools to figure out other aspects of the pages load/render time, but one I haven't found is to see if there are any poorly written jQuery selectors. An actual example I've found in production code was a selector taking 8.5 seconds on an iPhone 5, tweaking the selector, it now takes 50ms.

So I wrote a small wrapper for jQuery and jQuery.find that will log selectors, the number of items they find, and how long it took to find them. This allows you to check an entire page of widgets, modules and plugins to find any inefficient selectors.

You can find my initial version over at bitbucket, jQuery Selector Performance Monitor - jqspm

I'll be posting a follow up entry explaining more about its use in the coming days. Read the more detailed post.

Here is an example of the output, showing the selector used, the number of elements found, and time to find them.

TimeZones with Javascript

So I originally wanted to make a TimeZone selector similar to most OSes, you know, the world map with cities visible and you can select the one in your time zone. The thing I didn't want to do with this was all the work of building and maintaining a DB of cities and their GMT offset along with daylight saving rules.

So I looked around and found a time zone database, now hosted at http://www.iana.org/time-zones. I downloaded these and started writing my JS library to parse the list and generate the rules for all the time zone info. And thus jsTimeZone was born.

With this library it allows you to find the date/time in a time zone similar to most BE languages, so using 'Australia/Sydney', or 'Europe/London'. Being FE based, it will calculate based on the machines local date/time, so as long as it's accurate, this library should get the right result.

The added benefit to using this is when I go to make the map, I can just scan through the time zone info and I'll have city name and country all ready to look up GPS coordinates for and plot on my map. Though this one I'll generate once and include a static file in the repo to be used, along with a generation script in case new time zones popup down the track.

I have only tested a few of the more common time zones I use, let me know if any places don't return the right result.

You can also export the tz database from the module and use the returned JSON string when calling the module again.

Quick example of using this module

function queryTime() {
    var ausSyd=jsTZAll.offsetIn('Australia/Sydney');
    console.log('Aus - Syd: %o', ausSyd.getOffset());
    console.log('Aus - Syd: %o', ausSyd.date());
}

var jsTZAll=jsTimeZones('australasia', 'europe');
jsTZAll.init(queryTime);

Which will return

Aus - Syd: { hr: 11, min: 0}
Aus - Syd: Date {Fri Feb 22 2013 15:06:59 GMT+1100 (AUS Eastern Daylight Time)}

To see more, head over to the Bitbucket repo at jsTimeZone.

Software I use

So I've just done a fresh Format/Install, figured I'd document the tools I've grown a collection of over the years. DisplayFusion

Purchased. I couldn't live without this, task bars on each monitor, unbelievably useful. If you don't think so, you either don't have 20 applications open at once, or don't know what your missing. Download the demo and try it. The other features are neat, but I rarely use them.

Ditto

Free. Clipboard manager, store a ton of copies, and let you select the one you want to paste with a simple hotkey. I use this a ton while programming, but still find it useful when I'm not coding.

Take Ownership

Free. Registry hack that adds a 'Take Ownership' option to your right click menu. I use this when I copy files from a previous install of Windows, resets the permissions to me on my current install.

TeraCopy

Free. Windows Copy Replacement. Fantastic replacement, cut/copy files and folders like normal, and on paste it will queue your files, and perform the copy. Has error handling, so no more broken copies halfway through, and seems to copy slightly faster. Amazingly well done program, and some of their Android tools are pretty awesome too.

MSI Afterburner

Free. AMD VISION Engine Control Center replacement. Light weight, amazing fan control. This is so good I'd buy an MSI card over other comparable cards.

avast!

Free. Great and light weight antivirus scanner. Does a great job, all the features you want, none of the bloat you don't. It does offer to install Chrome, I select this option as I use both Firefox and Chrome, and it gives avast! some money from Google.

Defraggler

Free. Great defragmenter, see fragmented files, choose which ones to defragment. Super simple and works a treat.

VLC Player

Free. The best media player, light, powerful, great to use. Nothing comes close to VLC.

TightVNC

Free. Great Windows VNC server, small and fast with a multi-platform client(Java).

Irfanview

Free. The best image viewer ever. Reads almost everything, has some great adjustment controls and it's batch mode is simple and fast.

7-Zip

Free. Compression tool, reads/writes ZIPs, TARs, GZIPs and a range of other formats. Lightweight, does a great job.

MP4 with MP3 via HLS on Android - HTTP Live Streaming

If you have a MP4 with a AAC audio stream, both iOS and Android play it fine via the HTML5 Audio player over HLS. But if you use a MP4 with a MP3 audio stream, iOS plays fine, but Android won't play.

Turns out Android only plays as long as there is a video stream, so we need to create a video stream to go with our audio. The easiest method of this is with ffmpeg.

1. Create a blank 32x32 image, flat white, or black or any other color, I called mine image.jpg.

2. Create a looping video the length of the audio, or longer than the longest MP3 you'll be using.

ffmpeg -loop 1 -i image.jpg -vcodec libx264 -vprofile baseline -level 3 -t 360 loopVideo.mp4

This will loop the image.jpg for 360 seconds, and save the output as a H.264 video stream called loopVideo.mp4. The video is created at Baseline 3, so this will work on iOS 3+ devices.

3. Combine the MP3 with the loopVideo.mp4

ffmpeg -i loopVideo.mp4 -i "My MP3 Song.mp3" -c copy -map 0:v -map 1:a:0 -vprofile baseline -level 3 -shortest finalSong.mp4

This combines the video with the audio, it is a straight copy from both video and audio, so it occurs in 30ms or so. -shortest tells ffmpeg to clip the streams at the shortest one. If you made the video longer than all your MP3s, ffmpeg will clip the video stream to the length of the MP3 you're using.

Now your right to stream this via HLS onto Android. Let me know if you have any issues with this, or find better methods.

We're sorry, but something went wrong (500) - Kickstarter - Firefox

So went to Kickstarter.com as usual and was welcomed with this message "We're sorry, but something went wrong (500)". I figured they had some hicup and figured I'd come back later, a few hours later it was still having issues, but projects I was backing were tweeting about new updates, without a mention of KS being down. Then this morning, still no tweets about it being down and many more messages about KS updates, though the site still gave me a "We're sorry, but something went wrong (500)". So I opened it in Chrome, and it worked. Back to Firefox, I deleted all the kickstarter.com cookies and boom, Kickstarter was back in action.

Photos of Sydney Harbour Bridge from 1930s

Here are some scanned historical photos of the Sydney Harbour Bridge under construction.

August 7th, 1929. Sydney Harbour Bridge under construction. View from North side. They had used 19,200 tons of steelwork to this point.

Middle of April, 1928. Harbour Bridge from South side.

North side, ground level, nearly complete.

1930, Water view of South side of bridge.

1929, Cruiseship arrives with Bridge construction in background.

January 1930. View from Loftus St, Sydney. Harbour Bridge construction in background.

1930, Working 300feet above the water.

Portfolio: Corona Music Mexico

A music driven promotion website, launched down in Mexico recently. People can redeem vouchers from their Corona packages for credits that they can spend streaming or downloading music.

The initial design and HTML/CSS were created by a 3rd party Mexican agency, as I added functionality to the pages and further changes came through, I ended up re-writting much of the CSS and many of the HTML pages to solve cross browser issues, polish the design and facilitate functionality.

Most of the functionality is handled by Javascript, AJAXing to a BE which returns JSON. It features Music listings, creating/editing of playlists, sharing of artists and albums via Facebook, and the ability to stream 30sec and full length tracks via the Flash player.

I had to make heavy customisations to the Flash player to support next track pre-caching, streaming token retrieval, add to playlist, and purchase/download buttons for currently playing song.

I also built the simple HTML/CSS for a feature phone version of this site, and assisted the BE developers in supporting these devices for streaming.

You can see both these at descargas2.coronamusic.com and m.coronamusic.com

IE8 Facebook Login - "Object doesn't support this property or method" in all.js

So doing some more work on Facebook integration, we were posting to facebook.com/dialog/oauth/ and when it redirected back all.js was throwing an error 'Object doesn't support this property or method' at this line response={authResponse:FB._authResponse,status:FB._userStatus};

I couldn't find anywhere in all.js where it defined response, so a quick hack I added

var response={};

This solved the issue for me, hope this helps. If you know anything more leave a comment.

Facebook Javascript API - Post to wall - FB JS SDK

Working with the Facebook JS SDK I was having problems using the FB.ui to log the user is, and post a message to their wall. Main error was 'Permission denied to access property 'Arbiter'.'. I tried many different things such as developing this on a website that could be accessed from external sources, but the problem wouldn't go away.


So I wrote some code that used FB.login and FB.api, though after login/permission acceptance it would leave the XD Proxy window open on a blank page. Below is my code which will handle the XD Proxy window, login, permission acceptance and post to their wall, even on your local environment without external sources being able to access your site.


First we install the FB API

<script>
 window.fbAsyncInit = function() {
   FB.init({
     appId      : facebookAppId,
     channelUrl : 'http://'+ window.location.host +'/channel.html',
     status     : true,
     cookie     : true,
     xfbml      : true,
     oauth      : true
   });
 };

 (function(d){
    var js, id = 'facebook-jssdk'; if (d.getElementById(id)) {return;}
    js = d.createElement('script'); js.id = id; js.async = true;
    js.src = "//connect.facebook.net/en_US/all.js";
    d.getElementsByTagName('head')[0].appendChild(js);
  }(document));
</script>

Next we want to setup the function to handle the login/permissions.

function FBShare(title, link, picture, msg, caption) {
   FB.login(function(data) {
     postFBMsg(title, link, picture, msg, caption);
   }, {scope: 'publish_stream'});
 }

Here I call FB.login, setting scope to publish_stream which once the user is logged in will ask for permission to allow my app to publish to their wall.
Once login is complete and they have accepted my permissions, we call postFBMsg.

function postSocialMsg(title, link, picture, msg, caption) {
   var body={
     message: msg,
     picture: picture,
     link: 'http://'+ window.location.host +'/'+ link,
     name: title,
     caption: caption
   }

   FB.api('/me/feed', 'post', body, function(response) {
     if (!response || response.error) {
       // Error
     } else {
       // Successful
     }
   });
 }

Here I set the body var to contain our Facebook Wall Post details, we have the title(Don't Worry Be Happy) which will be linked to link, we have picture(Guy Sebastian pic), msg(Wall Post Message) which will appear under the Username(John Smith), title and caption(Guy Sebastian) which will appear under our Message to the right of the picture. If no Caption is sent it displays the link URL instead.

The final piece to our puzzle is to handle the closing of the XD Proxy window after login. It posts back to the calling window after actions are performed, I setup an event to capture these messages and close the window once it receives one. If you already are using postMessage between windows, you'll need to setup a conditional statement to only close the window if it's a Facebook message.

 $(window).bind('message', function(e) {
   e.originalEvent.source.window.close();
 });

So once we get a message via postMessage we close the window of the message sender.


This is how I'm doing it in my local development environment, inaccessible to the outside world, if you spot and flaws, or know of a better way of doing things please leave a comment.

Firefox 9.0.1 Error: uncaught exception: TypeError: args.shift() is null

So looking at a site we're developing this morning in FF 9.0.1 and some pages didn't work on some peoples machines, though worked fine on mine. After installing Firebug to see what was going on the pages would work fine.

I uninstalled Firebug and looked at the Firefox Error Console to find the following error.
Error: uncaught exception: TypeError: args.shift() is null


Was a weird one, after much poking around I found it's a poor handling of arguments supplied to console.log, hopefully this is a bug in FF9.0.1 and will be corrected in the next release. For now to quickly get things working I've just extended my console.log handler to set console.log=function() {} to cancel console.log functionality until it's fixed.

If it's not addressed in future versions I'll have to look into it further to detect if Firebug isn't active and do some alternate handling for development.

To work around this I've changed my code to this

if(window.location.search!='?debug=1') {
  top.console.log=function() { }
}

Now I just add ?debug=1 to URL and it uses built in console log so I can manage when it comes on.

Hope this helps, if you have further info please leave a comment.

Dungeon Siege III - GFX Files

GFX files are basically SWFs, to edit them rename them to a *.swf file and with a hex editor change the first 3 bytes from CFX to CWS, then any Flash Decompiler will happily open them.

I've opened them up and inspected the Actionscript which all worked though I'm yet to try editing one, my guess is you should just be able to change the first 3 bytes back to CFX and rename it back to an *.gfx file for it to work, though depending how it modifies the SWF it might break something extra that GFX files have that SWF doesn't.

Let me know if you've had any success, or more info, I'll update this post as I find out more.