Lab 7

Web Services

 

 

Introduction

 

This week we learned how web services can provide useful data to web applications. In this lab you will use a web service to get new relevant data, transform it via XSLT, and display it in your application.

 

Instructions:

 

  1. Copy your Lab06 new Lab07 directory. Your main file should still be called index.html

2.      Here is what you what you want to have when you are finished.  Suggested plan of attack for getting there is below (read it before you start!):

    1. Try to keep the functionality from your previous lab (e.g. don’t start over and delete things, but you don’t have to fix things either.  Talk to the instructor if you have a reason to modify this).
    2. There should be a button(s), link(s), or something to click or interact with such that when you click on one the following happens:
      1. You use AJAX to invoke a Perl/PHP proxy
      2. The proxy invokes a web service (you choose, see below)
      3. The web service returns XML to the proxy, which returns it to your HTML/JS application
      4. You transform that XML with XSLT (same general idea as last week, but see details below).
      5. You display part of the transformed XML in your page.
      6. All of this should happen within the same HTML page (e.g. the URL up top should not change)
    3. The web service must return different results based on different data in your database.  For instance, if you have people, you might show weather for their hometown when you click on one.  If you have products, you might show related pictures when you click on it. Thus, the same web service will be invoked, but different data from you will yield different results. This implies that your web service should accept some argument (talk to the instructor if you find a web service that you really want that doesn’t have this).
    4. It is perfectly fine to insert some new info into your database to make this lab work.  For instance, maybe you didn’t have zipcodes before, but now you need them for a service – go ahead and add them.
    5. You may choose any web service (some exceptions below) that returns XML and that permits you to use it for free non-commercial use (though you will likely have to register). Start this early! You may have to wait for a “key”, or it may turn out the service doesn’t provide what you need. Details:
      1. Tip: www.programmableweb.com to find web services. Click on “API Directory” – then look for search options.  Search for those with  protocol of  SOAP, REST, or GET. You want the data format to be XML. NOTE: not all the services have the “protocol” and “data format” fields stored on this site – so you might not want to filter on those at first.  Start by looking at the categories to see if you can find something that looks useful.
      2. Try to pick a service that adds real usefulness to your site.  Talk to the instructor if you need ideas.
      3. You may not use the two services used as lecture examples (weather.com for REST and Flickr for SOAP).  Note: Flickr also offers REST – if your application really calls out for photos, and Flickr is really what you need, you can try to convince the instructor to allow you to use Flickr via REST, subject to some additional requirements.
      4. You probably don’t want to use a mapping-based web service (we’ll add that later).
      5. Your must use a proxy that directly interacts with the web service using Perl or PHP (as the two examples in class did). For instance, you may not use one of the Google JavaScript libraries that directly adds a map and handles all the interactions behind the scenes.
    6. You must use XSLT to modify the XML that you get. It should transform the XML so that it fits in with your existing page. Don’t just display the XML that you get back! Style and appearance count here.
    7. As always, your code must be commented!
    8. On your default.htm page, provide some good, very specific comments for a user (e.g. the instructor) on how to use your site.  This should be specific to this lab – e.g. how can I see that your site meets the requirements for this lab? For instance, what to click, what to enter, etc.
    9. Default.htm must have these additional links:
      1. The “invoke link”: a link that directly invokes your web service, so that clicking on it shows the raw XML returned from your web service. This should not be to a saved XML file – instead, after clicking on this you should be able to modify the argument that is shown in the URL in order to see different XML results.
      2. Link to test1.xml. In the browser, clicking on this should show the result of the transformed XML.
      3. Link to test2.xml. In the browser, clicking on this should show the raw XML.
  1. Suggested plan of attack (see also helpful info at end)
    1. Do NOT follow the requirements above in order. Instead…
    2. First identify a web service and register (if necessary).
    3. Figure out how to access the web service via your browser (or a Perl proxy, if it’s SOAP).
    4. In default.htm, add the “invoke link” that is described above.
    5. Using the “invoke link”, save a sample XML result from the web service – call it test1.xml. You will have to turn this in.
    6. Write a XSLT transformation and test it on test1.xml by adding the XML processing instruction up top. Don’t forget to verify your XSLT file alone by loading it in a browser.
    7. Copy test1.xml to test2.xml, and remove the XML processing instruction from the top of test2.xml.
    8. Write the JavaScript to load the XML (from test2.xml), apply the XSLT to it, then insert some of the result in your page.
    9. Modify your JS to invoke the Perl/PHP proxy to get the dynamic XML data (instead of fetching the static XML), then transform via XSLT and display on your page.
    10. In default.htm, add instructions AND links to test1.xml and test2.xml (see requirements above).
    11. Test and make sure everything still works!  You’d be surprised how often things get accidentally broken that you were sure worked.

4.      Ensure your page works with Firefox. Having it work on IE is encouraged but not required.

5.      Ensure all your pages validate and that you have met all requirements.

     

NOTE: all HTML files must validate as XHTML without errors for full credit.  The penalty for a file that does not validate is 10%.

Helpful stuff

  1. See the plan of attack above!
  2. General debug strategy:
    1. First, run Perl from the command line and make sure it works (see below).  Include actual arguments so you can see it’s really working.
    2. Second, run Perl via the webserver directly, e.g. something like
      http://www.mXXXXXX.it452.cs.usna.edu/Lab07/query.pl?q=dogs&type=4
    3. Finally, try it all from HTML via the browser.  If this doesn’t work
      1. Look first at the Error console!
      2. Use window.alert() to see what is happening.  In particular, check what URL you are actually using to invoke Perl.
      3. Did I mention looking at the Error console?
  3. As always, the “Error Console” in Firefox is invaluable.
  4. Perl is easier to debug via the windows command line. You should first check for syntax errors by just “checking” the file:
         perl –c -w myquery.pl
    Once that works run it for real:
         perl -w myquery.pl
    If needed, you can provide CGI arguments like this:
         perl –w myquery.pl “query=dogs&type=2”
  5. On the command line, database errors will usually appear automatically.  From the browser, they won’t – instead insert this as a debug:
         print $DBI::errstr;
  6. Useful SQL examples:
    1. CREATE TABLE comments (USER_NAME VARCHAR(20), TIMESTAMP DATETIME, COMMENT MEDIUMTEXT, PAGE INTEGER);
    2. INSERT INTO comments (USER_NAME, TIMESTAMP, COMMENT, PAGE) VALUES ('Jamie', '2006-09-27 11:30:00', 'hi', 2);
    3. CREATE TABLE topics (TOPIC VARCHAR(20), OWNER VARCHAR(20));
    4. INSERT INTO topics (TOPIC, OWNER) VALUES ('Cars', ‘Jamie’);
    5. Note: ID is automatically created with the table, no need to specify it.
  7. Do not use the die() function alone – the error it generates can’t be seen from your web page.

 

Deliverables

1)      All of your files for Lab07 should be in a folder called "Lab07" (without the quotes) on the Web drive.

2)      Your main page for Lab07 should be called "index.html" (without the quotes) and placed inside the folder Lab01.

3)      Your instructor will assume that your web pages are viewable at http://www.mXXXXXX.it452.cs.usna.edu/Lab07/index.html where XXXXXX is your alpha number. You should check that this URL is viewable and that everything works correctly from a computer where somebody else is logged in. If you've goofed and linked to a file on your X drive, this will help you catch it!

4)      All files must be complete and saved to your Web drive before you submit the hardcopy of your assignment. Do NOT modify your files after you have submitted your assignment.

5)      Turn in the following hardcopy on or before the due date, stapled together in the following order (coversheet on top):

a)      A completed assignment coversheet.  Your comments will help us improve the course.

b)      The first page of the output of the W3C validator, as run on the final version of your Lab07 HTML that does the new transformation.  This should show that your document correctly validated, but turn it in anyway if you can’t get your page to validate. 

c)      A printout of the source to your Lab07/index.html file (or other HTML file that has the new transformation).   Truncated lines are not acceptable – use Crimson Editor vice Notepad if needed for printing.  You could also paste into Microsoft Word etc. if needed.

i)        NOTE – turning in just the JavaScript fragment that does the transformation and insertion is acceptable if you wish.

d)      Printouts of external JavaScript files, if any.

e)      Printout of your Perl/PHP file that acts as a proxy

f)        Printout of sample.xml returned by the web service you use. Just the first page is okay, if long.

g)      Printout of your XSL file