Macaw

Stop writing code. Start drawing it.

More information on:

Advertisements

Web Scraping with PowerShell

In PowerShell v3 you have some new useful cmdlets that allow you to download and parse a website.
The code in this post will demonstrate very basic scripts that could get you started with Web Scraping.

If you don’t know if you have PowerShell v3, use this command to find out:

get-host

The first script to get you started with web scraping:

$site = Invoke-WebRequest -UseBasicParsing -Uri www.bing.com
$site.Links | Out-GridView

This will give you all the links from the given website in a gridview.

The next script will give you all the email addresses that are in a mailto: anchor:

$site = Invoke-WebRequest -UseBasicParsing -Uri www.mywebsite.net
$site.Links | foreach {
if ($_.href.ToLower().StartsWith("mailto:")) {
$_.href.SubString(7) | Out-Default
}
}

By coincidence the ‘mywebsite.net’ has anchors using the mailto: prefix.

The last script is a very cool script from StackOverflow where I just modified the url to make sure the script works in several European countries:

function Get-FlightStatus {
     param($query)
$url = "http://www.bing.com?cc=us&q=flight status for $query"
$result = Invoke-WebRequest $url
$result.AllElements |
        Where Class -eq "ans" |
        Select -First 1 -ExpandProperty innerText
}

Use it like this:
(to test you can just paste this after the function in Windows PowerShell ISE )

Get-FlightStatus LH3102

It will give you a result similar to this:

Flight status for Lufthansa 3102 
flightstats.com · 2 minutes ago   

Departing on time at 5:35 PM from HAM 
FROMHAM 
Hamburg5:35 PM 
12/30/2012Terminal 2 
TOVIE 
Vienna7:05 PM 
12/30/2012

PS C:\>

Don’t forget, web scraping can be illegal!

Have fun 😉

Take a look at “Web Scraping with Perl” and the PowerShell tag.

JsFiddle: a playground for web developers

Today I came across a great website for testing JavaScript. It’s a place where you can test, edit and share code and even debug code from other people.

JsFiddleis a playground for web developers, a tool which may be used in many ways. One can use it as an online editor for snippets build from HTML, CSS and JavaScript. The code can then be shared with others, embedded on a blog, etc. Using this approach, JavaScript developers can very easily isolate bugs. We aim to support all actively developed frameworks – it helps with testing compatibility

Have fun 😉

link ] [ link2 ]

The evolution of the web

The web today is a growing universe of interlinked web pages and web apps, teeming with videos, photos, and interactive content. What the average user doesn’t see is the interplay of web technologies and browsers that makes all this possible.

Over time web technologies have evolved to give web developers the ability to create new generations of useful and immersive web experiences. Today’s web is a result of the ongoing efforts of an open web community that helps define these web technologies, like HTML5, CSS3 and WebGL and ensure that they’re supported in all web browsers.

The color bands in this visualization represent the interaction between web technologies and browsers, which brings to life the many powerful web apps that we use daily.

View the timeline.

[ source ]

HTML5 support for Visual Studio 2010 Editor

Today, a faction of the Web Platform and Tools team, spearheaded by Mads Kristensen, is pleased to announce the Visual Studio Web Standards Update. This adds better support for HTML5, CSS3 and new JavaScript features to ALL versions of Visual Studio.

More info and the download link here.

[ source ]

Modernizr: a Javascript library to detect emerging web technologies

Modernizr is a small and simple JavaScript library that helps you take advantage of emerging web technologies (CSS3, HTML 5) while still maintaining a fine level of control over older browsers that may not yet support these new technologies.

[ source ]

Internet Explorer 9 (IE9) Test drive – developer preview

Download the Internet Explorer Platform Preview and discover the new JavaScript Engine and the HTML5 and CSS3 support in this early preview build. This is only a preview so the UI will change and the engine is not at its final stage yet (keep this in mind).

Glad to see that Internet Explorer 9 welcomes HTML5 and CSS3!

[ source ]