±Forensic Focus Partners

Become an advertising partner

±Your Account


Username
Password

Forgotten password/username?

Site Members:

New Today: 0 Overall: 35896
New Yesterday: 1 Visitors: 150

±Follow Forensic Focus

Forensic Focus Facebook PageForensic Focus on TwitterForensic Focus LinkedIn GroupForensic Focus YouTube Channel

RSS feeds: News Forums Articles

±Latest Articles

±Latest Videos

±Latest Jobs

downloading a web page

Computer forensics discussion. Please ensure that your post is not better suited to one of the forums below (if it is, please post it there instead!)
Reply to topicReply to topic Printer Friendly Page
Forum FAQSearchView unanswered posts
 
  

PaulSanderson
Senior Member
 

downloading a web page

Post Posted: Jan 23, 17 01:51

I have an odd little background task that is vexingme slightly.

I want to download a web page that displays certain offers that can be filtered by a date range and location and then the resultant offers are grouped and displayed 10 per page. You can view each group of pages using pre/next buttons or selecting page 1 of x from a drop down menu.

The web page does not use html variables ie on filtering the url remains the same.

I want to programatically select a date range and then select and save each page - an example of some of the underlying code is:

<div class="offer-nav">
Skip to:
<select onchange="webpage.offer.goToPage(0,'', $(this).val(), $('.filterLocn').val(), $('.fliterDate').val(), 1234);">
<option value="1" selected="selected">
Page 1
</option>
<option value="2">
Page 2
</option>
<option value="3">
Page 3
</option>
</select>
</div>

Does anyone have any pointers to a resource that might explain how I can go about this?

This is a bit of an intellectual exersize as I have saved the pages I want manually (there weren't too many) but it got me wandering how one would automatically save a web page where the data behind is the result of a filter of this sort.
_________________
Paul Sanderson
SQLite Forensics Book
www.amazon.com/SQLite-...entries*=0

Forensic Toolkit for SQLite
sandersonforensics.com...for-SQLite 
 
  

JDCoulthard
Senior Member
 

Re: downloading a web page

Post Posted: Jan 23, 17 02:43

Hi Paul,

You could take a look at Python and Selenium for this exercise.

If you identify the form fields by XPath, you can change variables and submit forms. You can then either save the resulting HTML or parse it with something like BeautifulSoup.

HTH

James  
 
  

PaulSanderson
Senior Member
 

Re: downloading a web page

Post Posted: Jan 23, 17 03:09

Thanks james

Just had a very brief look - seems to be just what I want.

P
_________________
Paul Sanderson
SQLite Forensics Book
www.amazon.com/SQLite-...entries*=0

Forensic Toolkit for SQLite
sandersonforensics.com...for-SQLite 
 

Page 1 of 1