Extract Table from HTML

[Jon_Kokott]

Championship

Created a website to easily share data with others.

Summary:

Now that I've gathered, processed, and displayed data, its time to share it with the world.  I've chosen to upload the html website generated last round to a server.

For this round I have:

• Created a website: http://fantasyfootballexpert.heliohost.org/SeasonRB.html
• Automated the entire process of downloading and processing web content, to creating and uploading/updating the final result webpage.
• The entire page can be updated in about 15 minutes.
• Easyest to use share method for others, just email/post the link.
• Added links to ESPN player profile webpages.
• Further refined various aspects of the webpages.
• Software uses same engine (built on a LVOOP classes) to compress/uncompress html documents.

Steps to execute

1. Open CodeMadnessChallenge.lvproj
2. Open "Championship Main.vi"
3. Run "Championship Main.vi"
4. Wait a few moments, there is now a few dialogs that will update you as to what the code is doing while data mining/parsing/processing is going on.
5. Cancel the operation where the results are uploaded, I won't be giving out root access to the website, but I assure you it does work.
6. Open the local copy, or better yet, goto: http://fantasyfootballexpert.heliohost.org/SeasonRB.html

Screenshots:

VI Snippet:

Third Round

Manipulate data using another programming language, and display it.

Summary:

Are you a fantasy football enthusiast like me?  Have you ever wondered what expert gives the best advice?  I have, and I wanted to do something about it.  With this code you can take a look at how each expert does on a weekly basis for the running back position.  I used the ESPN.com website for the source data as its one of the more popular fantasy football sites around.

This code will:

• Rank expert fantasy football analysts and display data in html.
• Give secondary pages with weekly performance comparisons for each expert.
• Graphs weekly performances in Matlab.

Steps to execute

1. Open CodeMadnessChallenge.lvproj
2. Open Final4bit.vi
3. Run Final4bit.vi
4. Wait a few moments, the code only takes about 30 seconds to generate the html, but I didn't have time to put a progress bar in.
5. copy the created html path into a web browser to explore the results.

note: code requires "NI html toolkit" for this round.

Screenshots:

Experts scored by week

All the links can be followed.  This will send you to the page below (different for each week/expert) and view a detailed breakdown.

Example week breakdown for one expert:

Matlab Graph

VI Snippet:

I know this is not very complicated, but interfacing with MATLAB is one of the easyest things to do in Labview.  This demonstrates the ease of passing data in and out of labview.  This code runs very quickly, and there are a whole variety of interesting MATLAB graphs and data analysis tools that can be very helpful.

Code is attached below. (HTML_Extration_LV2010.zip)

I have predownloaded all the websites for you.  this should not mine through the internet unless you delete the include files, in that case it will download alot of data.  This will greatly extend the time to create the html analysis site.

DVR Reasoning:

DVRs allow you to access individual data elements in the array without the need to transfer the entire array worth of data, or resize/manipulate the array by value.  Becauase DVRs each reside in their own memory space, they will not interract with each other the way array manipulations do.  Passing arrays between VIs will also result in data copies, where as with a DVR the data is shared between the VIs.

Steps to execute

1. Open CodeMadnessChallenge.lvproj
2. Open Main.vi
3. Run Main.vi
4. Create data from a website.
5. Run a test to compare DVR arrays to by value arrays.

Screenshots:

VI Snippet:

First Round:

Gather Data from a web page.

Summary:

• Recursively search html tags for a table
• convert table to 2D string array
• Parser exandable to search by specific html tags.
• Creates a LVOOP tree structure in memory replicating html structure

Steps to execute

1. Open CodeMadnessChallenge.lvproj
2. Open Main.vi
3. Run Main.vi
4. Select the example page from the enum selector.
5. Watch as a common engine parses various html tables.

Screenshots:

VI Snippet:

[Jon_Kokott]

This code may not run unless you have installed LV2010 sp1.  It uses LVOOP property nodes which can/will crash labview before 2010 sp1

.

[G-Money]

Do you happen to know which ones? Was it a bug?

[Jon_Kokott]

255982

Child Class Property Node does not output until deleting and rewiring any part of the VI
This issue pertains to Object-Oriented Programming in LabVIEW. Essentially upon creating a Parent class and Child class, we are unable to read the child class properties in the main VI through property nodes unless we have changed the VI recently (i.e. unwire & rewire).

Workaround: The workaround for this issue is to use subVIs created from 'VI for Dynamic Dispatch Template.' Within the subVIs you are able to unbundle and bundle the object.

Reported Version: 2010 32-bit Resolved Version: 2010 SP1 32-bit Added:12/31/2010

[Jon_Kokott]

anywhere i use a LVOOP property node it could crash.  you should be able to open the VI in 2010, just dont run it unless its OK for labview to crash (LV2010 pre SP! only)

once again, it runs fine in LV2010 SP1

[Jon_Kokott]

The tests aren't working properly for suggested values.

They are behaving much differently depending on whether I use LV2010 64 bit vrs LV2010 32 bit.

[Jon_Kokott]

My Intention for round 2 was to show the difference between the way labview stores data as an array or as a DVR.  Unfortunately the tests are not doing such a good job of that.

[Jon_Kokott]

I used standard scoring for comparing the performance of each expert (6 pts/TD, 1 pt/10 rushing or receiving yards, -2 for fumbles lost)

The results are weighted in the following way

(Player scored points) x (50 - rank) where rank starts with 0.  so there is a 50 point multiplier for the #1 ranked player each week, and a 40 point multiplier for the #10 player.  I normalize the results so that they are near 100 (the typical total of a fantasy football week.)

[Jon_Kokott]

One thing to note:

I did not use any of NI's (or anyone else's for that matter) html generation tools.

I have written my own html scripter which is significantly more expandable (object inheiritance, tree structures) to allow for things like tables within tables, hyperlinks, images, and anything else that belongs in an html.