Perl Help If Possible?

skirrow posted this at 09:06 — 12th July 2000.

Joined: Jul 1999

Hi,
I'm writing a little cgi script. I've done most of the logging part. But need to do the log reading now and I'm stuck. If anyone could help me I can probably give them some banner impressions or something!
Here's the problem:
I have a log file (text) that is in the following format:
Username|ip address|referring url|web browser|Date

What I want is to be able to display all the results in a web page in the following way.

Top 10 most used usernames
Top 10 most used ip addresss
Top 10 referreing urls
Top 10 web browsers
List last 100 logins.
List last user to login.
And that's it.

How do I do this in Perl?

Thanks a lot

Neil

DreamStreamUK - The webmasters network.
The CGI-Index.com - Over 200 categories of CGI resources.
Web-ScriptsUK - CGI Scripts for the internet novice.

anti posted this at 15:10 — 12th July 2000.

They have: 453 posts

Joined: Jan 1999

Just some common tips...

If you use "|" seperated tables, always end the line with an "|".
There is a very good reason for it, but I used to many RDBMS to remember right now.

Use the system if you can.
Under UNIX tail, sort, uniq, etc are your friends.

Last 100 logins:
tail -n 100 LOGFILE > LAST100
Last login:
tail -n 1 LOGFILE > LAST1
Top 10 IPs:
cut -d"|" -f 2 TOP10IP

If you want to do this in pure perl you would use a similar approach.
Depending on the logfile size you could do it in memory.
But why bother. Just let the shell to all the work ...

If you have _tried_ to do it and find some specific problems ask again.
Nobody can/will help you with "I need the script, please write it."

ciao
Anti
ps:
If you really can't do it, but need it ... buy it ... my price is 250US$/hour (discount for TWF members included).

skirrow posted this at 15:40 — 12th July 2000.

They have: 56 posts

Joined: Jul 1999

Hi,
Thanks anti!
Okay. Maybe this is asking a bit much! But I havn't programmed seriously for over a year so I'm extremely rusty!

Could you possibly just write the top 10 IPS section?
All of it, including how to print them.
It can't be much code looking at what you showed me!

Thanks a lot!
Neil

DreamStreamUK - The webmasters network.
The CGI-Index.com - Over 200 categories of CGI resources.
Web-ScriptsUK - CGI Scripts for the internet novice.

anti posted this at 09:52 — 13th July 2000.

They have: 453 posts

Joined: Jan 1999

But beware I have no perl at hand, so it all commes from my head.
(And I'll go via the shell, since Idon't want to write a sorter ...)

#!/usr/bin/perl

$INFILE="LOG";

system( ( "/bin/sh", "-c", 'cut -d"|" -f 2 <'.$INFILE.' | sort  | uniq -c | sort -n | tail -n 10 | cut -f 2 >TOP10IP' ) );

open( TOP10IP, "<TOP10IP" );
while( <TOP10IP> ){
	print $_;
}

Note:
This is not very fast or resource friendly,
but I assume this is for your private datamining only anyway, so it will be OK.

And yes, I thought it would be more code ...

Anti

skirrow posted this at 10:19 — 13th July 2000.

They have: 56 posts

Joined: Jul 1999

THANKS A LOT!!!!

Neil

anti posted this at 09:43 — 14th July 2000.

They have: 453 posts

Joined: Jan 1999

Did it work ?!