--------------------------------------------------- Lab #10 -- Code coverage; twill; Jinja2; JavaScript --------------------------------------------------- CSE 491, Nov 5th, 2009. I will have FileServer grades to you tonight. (NOTE: both figleaf and twill are open source and freely available, not to mention written by Yours Truly.) Code coverage analysis with figleaf ----------------------------------- figleaf is a tool that, when run on Python source code, tracks which lines of code are actually executed. It then lets you annotate your Python files with HTML output. Its primary use is to "watch" your code-under-test to see what lines of code are actually executed. Code that isn't executed is **definitely not tested** and is a good candidate for your next test writing effort. Even with 100% code coverage, you won't be doing complete tests; consider: :: def f(a, b): if a: # path 1 else: # path 2 if b: # path 3 else: # path 4 You get 100% code coverage by calling: :: f(True, True) f(False, False) but you're only testing paths 1,3 and 2,4, not paths 1,4 or paths 2,3. For that you'd need branch coverage, which isn't yet possible with Python. To run figleaf on adriatic, do your standard :: source ~ctb/python-env.csh Then run whatever Python code you want to run like so: :: figleaf e.g. :: figleaf tst.py This will record what lines of code are run. Finally, run figleaf2html: :: figleaf2html which will produce a directory html/. This directory contains the HTML output of the code coverage run. Some practical tips ~~~~~~~~~~~~~~~~~~~ Practically speaking, you will want to do two things: first, you want to run figleaf **on your tests**, inside of nose. To do that, instead of running :: nosetests do instead :: figleaf /user/ctb/install/bin/nosetests (again on adriatic). Also, you'll probably want to put the output directory somewhere where you can access it remotely, e.g. your ~/web/ directory. To do *that*, first make sure you have a world-readable 'web' directory in your home: :: mkdir ~/web/ chmod -R a+rx ~/web/ and then tell figleaf2html to put its output there: :: figleaf2html -d ~/web/figleaf and make the 'figleaf' directory world-readable :: chmod -R a+rx ~/web/figleaf Now your figleaf output should be available as :: http://www.cse.msu.edu/~nermal/figleaf/ If you get the response "permission denied" when you go to the Web page, then you need to do the two chmods up above. (I can give you some code to get you started with testing FileServer; someone remind me...) Functional Web testing with twill --------------------------------- 'twill' is a simple, domain specific language for Web testing, implemented in Python. See http://twill.idyll.org/ for basic info, and http://twill.idyll.org/commands.html for a full list of commands. The basic idea behind twill is to **enable** *functional testing* of simple Web apps. For example, :: >> go http://localhost:5000/ >> code 200 visits that URL and asserts that an HTTP status code of '200' was returned. :: >> go http://localhost:5000/form.html >> formvalue 1 name "titus" >> submit >> code 200 fills in the input box named 'name' with titus on first form on the page, and verifies that upon form submission an HTTP status code of 200 is returned. The command :: >> find text asserts that 'text' is found on the page, and the code :: >> url /form.html asserts that the current URL has '/form.html' in it. You can use twill interactively (... voila) or you can create little twill scripts and run them as :: twill-sh <-- Hide your JavaScript from non-JS users (`page `__, `source `__) : :: Load JavaScript code from another file (`page `__, `source `__ and `JS source `__) : :: Grouping JS statements into blocks with squigglies (`page `__, `source `__) : :: Basic JS prompts (`page `__, `source `__) : :: alert("something happened!"); confirm("may I delete all your files?"); prompt("some question","defaultvalue"); Functions, head/body separation, and element events (`page `__, `source `__) : ::
'undefined' implicit conversion to strings, & errors (`page `__, `source `__) : :: document.write("this.bar is: " + this.bar) document.write("

foo is: " + foo); document.write("

and hello, world!"); (Go look at the Firefox error console.) Simple object-oriented programming, and iteration over bags o' properties (`page `__, `source `__) : :: function some_object_method(param) { document.write("hello! This is SomeObject's other param: "); document.write(this.what); document.write("

and this is my method param:"); document.write(param); } function SomeObject(what, other_param) { this.foo = "hello"; this.what = other_param; this.method = some_object_method; } ///// x = new SomeObject("world", 2); document.write(x.foo + " " + x.what + "

"); x.method("argument to SomeObject.method"); for (z in x) { document.write('

z is ' + z); document.write('

x[z] is ' + x[z]); } Working with HTML elements (`page `__, `source `__) : ::

New title: