Automated Testing with vncdotool (Not Headless, but Hairless)

The headless testing problem is a thorny one and impacts the fundamental architecture of the test automation platform I'm working on. If we can't find a solution, we might not be able to use the private cloud nor parallelize our testing effectively.

Part of the problem are the tests themselves that still rely on SendKeys() and MouseMove(). Two aspects of remote interaction that are insecure and therefore disallowed by Microsoft from Windows Vista onwards. This means that we have to rely on hacks like the VIX API and the one that I am about to document below.

There are two widely-used technologies for remote interactive connectivity on the Windows platform - the Remote Desktop Client (RDC) and VNC. The first is entirely proprietary while the second offers a number of open-source implementations (and some proprietary, full-featured, commercial implementations). The RDC implementation requires access to the VIX API which in turn relies on proprietary, privileged access to the Windows environment. The VIX API is used to access whatever proprietary VNC-like console app that VMWare uses on their stack. VIX also exposes RDC hooks. (I haven't used VIX RDC, I've just scanned a few documents so it's possible I got this all wrong - at any rate it's proprietary and I prefer open source alternatives). VNC, by contrast uses the Remote Frame Buffer protocol (RFB) which is an open standard across Windows and *NIX hosts. It is more limited than RDC in many ways but it should serve our purpose well enough.

After trying RealVNC - VNC® Open and TigerVNC, I settled on TightVNC which was easy to setup and had more administrative features than the other open source tools.

The biggest advantage of VNC is that it offers an interactive session for remote GUI testing, but unlike RDC, you don't need to keep a WIndow open for it to work. It is also based on an open standard which means that there are a number of open source clients available to use it programmatically.

There is another promising solution out there called Spice, that I would have liked to play with but I haven't had the time to take a serious look at it.

Another advantage that VNC has is that the VNC server acts as a splitter so you can give view/write access to a number of other users, unlike RDC.

I first tried ruby-vnc -  A library for interaction automation of servers via VNC for Ruby. However the documentation was fairly limited and I didn't want to wade through the source code to figure it out. Then I tried Net::VNC. This is a Perl library, much like ruby-vnc but it offered a little more documentation. However after hours of testing I found it highly unreliable as it would stop sending keystrokes to the target machine without any errors or explanation.

Finally I tried vncdotool. This is a command line VNC client written in Python. It is still in development but it is far more robust than any of the other tools out there. It is simple to use and extremely intuitive. You can daisy-chain all the mouse and keystroke interaction you want.

I had to disable Ctrl-Alt-Del for logon using the local security policy editor when I was trying the Ruby and Perl scripts because they didn't have a very elegant solution for sending complex key-strokes but vncdotool supports complex keystrokes quite well so we can use it without disabling this feature, if necessary.

Installation is fairly simple - it requires Python and a Python library called twisted that you can install using yum on Linux. You then download the vncdotool tarball, extract it and install it using 'python setup.py install' as normal. This adds it to /usr/bin/ which is already in your PATH so you can use it from wherever you are. I used the following command line from Linux to run the tests -

vncdo -s xxx.xxx.xxx.xxx -p pass --delay=3000 type "Administrator" key tab type "Passw0rd" key enter key ctrl-esc key up key up key right key enter type "cmd" key enter type '"C:\Program Files (x86)\SmartBear\TestExecute 9\Bin\TestExecute.exe" C:\AutomationWrapper\Engine\SideBar\Tests\app.pjs /p:RealTime /u:SideBarSuite /rt:SideBarconfidence /exportlog:C:\AutomationWrapper\Engine\SideBar\Tests\Log\SideBar_confidence_vnc.mht /r /e /SilentMode' key enter

The main thing to note is that the sendkeys and mouseclicks were both executed as they would be in an interactive session, unlike PSExec, PAExec or Winexe.

Using the tool is straighforward -

If I were to break down the example I used above for the actual test into key pairs you'll see the following -

vncdo -s xxx.xxx.xxx.xxx -p pass --delay=3000

• vncdo is an alias for vncdotool (you can use either command)
• I'm connecting to the TightVNC server at IP address xxx.xxx.xxx.xxx
• I have set a password on the VNC Server, which is 'pass'
• I want a 3000 millisecond delay between each of the following commands

type "Administrator"

• The cursor focus defaults to user login so I send it 'Administrator' as the username string

key tab

• I send it a tab command to move the cursor to the password field

type "Passw0rd"

• I send it the password string

key enter

• I send it the 'Enter' key to login

key ctrl-esc

• I send it a keyboard shortcut to invoke the start menu

key up

key up

key right

• I navigate by clicking the up-arrow twice and the right-arrow once to select the 'Run' command from the Start menu

key enter

• I hit 'Enter' to launch 'Run'

type "cmd"

• I type in 'cmd' into the 'Run' box

key enter

• I hit 'Enter' to launch the command prompt

type '"C:\Program Files (x86)\SmartBear\TestExecute 9\Bin\TestExecute.exe" C:\AutomationWrapper\Engine\SideBar\Tests\app.pjs /p:RealTime /u:SideBarSuite /rt:SideBarconfidence /exportlog:C:\AutomationWrapper\Engine\SideBar\Tests\Log\SideBar_confidence_vnc.mht /r /e /SilentMode'

• I send it the command for the test

key enter

• I hit 'Enter' to launch the test

The test was invoked remotely on a Windows machine from a Linux client.

vncdo also lets you take screen captures of the target machine. It supports four formats - BMP, GIF, JPG/JPEG and PNG. The format is determined by the file extension you use in the command -

vncdo -s xxx.xxx.xxx.xxx -p 1234 --delay=1000 capture screen.jpg capture screen.png capture screen.gif capture screen.bmp

• In this example I captured all four formats with a one-second delay between them
• It is quite slow and can take a while (up to a few seconds) to complete
• The file sizes were as follows -
• BMP - 1406KB
• GIF  -   483KB
• PNG -   259KB
• JPG -     25KB

The first screen capture was just a blacked-out image in every case. Increasing the delay or changing the format didn't seem to help either. If I just wanted a single file format I am forced to use the following command (do the capture twice) -
vncdo -s xxx.xxx.xxx.xxx -p 1234 --delay=1000 capture screen.jpg capture screen.jpg