delete files from a directory and db using php, mysql and crontab

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Plan:

1. set up a crontab to run #2 nightly.
2. write a script that will do these things:
a. establish today's date
b. establish a date two days hence
c. create an array of files that will expire in two days
d. email this information to the administrator email (and here's the tricky part) and to the owner of the files in question (there are n clients, but only one administrator who gets the information).
e. delete all files from the database that expire today
f. delete all files from the DIRECTORY that expire today
g. delete all files from the DIRECTORY that expired six months ago (to clean up, just in case)

Files:

zerocattle.com/examples/crontab.txt
zerocattle.com/examples/expiredfiles.phps

Knowing that:

1. all files are in this format: client##_filename.ext
2. $file_expiry is EDITABLE by the admin and in some cases the clients themselves
3. there are n clients, but one admin

What's the best way to do this?

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

Is there something wrong with the linked files?

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

... wrong like you can't reach them or wrong like they don't work?

They are heretofor untested. For the most part, I'm looking for confirmation that I am doing it right, that there ISN'T a simpler way, or any glaring errors. I'm not sure how to test the older file deletion, or if this is even the right way to go about it.

The crontab I'm happy to report, works fine, lol. I got a lovely error message last night that it couldn't find expiredfiles.php (I don't have it in a place where it can work until I'm sure how I'm going to go about it), so that much is right.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

While talking to the client, what I really need to do is delete files by file-name for that date.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Further discussion, and I've run into a bit of a mental block -- notably, when a client is deleted from the db, I need to delete all the files in the directory associated with that client.

Otherwise, the rest of the script should work as intended.

I don't want to just hope they don't lose clients, so if you wouldn't mind taking a look at the script again, would someone be able to unravel the knot I've put in there?

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

I'm having a hard time because I don't know what your db looks like or your directory structure.

And why are you doing this:
foreach ($email_rst['id'] AS $ID) {

Mark Hensler
If there is no answer on Google, then there is no question.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Ah, okay.

DB - two tables, one for files, the other for users.

files contains: file_id, id, file_name, file_size, file_description

users contains: id, firstname, lastname, company, email, username, password

one directory for all the client files, each file is named "client" . $id . "_" . $filename

Pretty simple, right? Nothing unexpected in there?

I was thinking with the foreach to loop through each id that belongs to a client, to check to see if that id exists... oh, I see what you're saying.

So help me here, I have files named clientID_file_name, where ID is the id of the client and file_name is the rest of the filename, including the extension. When a client is deleted, s/he's removed from the db, but the files aren't eliminated from the directory.

Perhaps that's the right place to do it, then, while I have a valid ID and I don't need to check an array...

Sorry for the confusion! Clearly it's because I hadn't thought it all the way through.

SO! Then my question is how do I delete a file from a directory, *knowing its name* (since I can make an array of all files for that id before I delete it, right? Right out of the db).

Wow, that's a lot simpler question. Anyone have the answer?

He has: 1,758 posts

Joined: Jul 2002

woooaah... complicated stuff... thats way beyond me.

Our developers here have written an automated system that deletes customers hosting accounts after a certain time if payment hasnt been received, however everything is intricatly linked, I think we use the domain name as a reference as every service we sell has to have a domain name associated with it... Your files need to have a unique ID that can be linked up to a record in the DB. I'm sure that would make things simpler.

Andy

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Thanks, Andy -- they do. $file_name is unique. If you upload the same file to the same client, they are named in successive order -- i.e. client28_thisfile.jpg, client28_thisfile01.jpg, client28_thisfile02.jpg. All the files are in the same directory per client request, which is part of the problem. If I had separate directories for each client, I could just delete the directory when the client was deleted.

The expired files should delete fine, it's just the files attached to expired CLIENTS that are a bit confusing to me at the moment, but I think once I know for sure what function will delete files from a directory (unlink works?) then I will be able to have those files deleted when I delete the client.

I think.

He has: 1,758 posts

Joined: Jul 2002

basically i think for each expired customer you need to loop through every file in the directory and delete those that contain the customer id in their file name... so a loop inside a loop kinda thing should do it.

the function you need is delete a file is

<?php
unlink
($filename)
?>

to match the customer id in the file name you need to do

<?php
ereg
(\"$expired_customer_id\", \"$file_name\")
?>

so to delete a file that matches you do:

<?php
if (ereg(\"$expired_customer_id\", \"$file_name\")) {
  unlink(
$filename);
}
?>

I hope thats some help... I hope i havent misunderstood what your trying to do...

Andy

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

That is helpful, thanks. Unlink seems so innocent, when really it's a killer. Smiling

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

not tested, but should help with the theory...

<?php
// includes functions and common variables
include \"/path/to/common.php\";

$upload_dir = '/path/to/uploads';

// connect to the database
connectdb();

// check file table for files with today's date
// this is done first, as we don't want to email the client with these files
// included in the 48 hour list
$deletenow = mysql_query(\"DELETE * FROM files WHERE file_expiry=CURRENT_DATE\");


$result = mysql_query(\"SELECT email, id FROM users WHERE usertype='c'\");
// loop through each client
while (
$user=mysql_fetch_array($result)) {
    // create an array of client id's
   
$client[] = $user;
   
    // reset
$message
   
$message = '';
   
   
$result = mysql_query(\"SELECT * FROM files\"
                .\" WHERE file_expiry<=DATE_ADD(NOW(), INTERVAL -2 DAY) AND id='
$user[id]'\");
    while (
$result = mysql_fetch_array($result)) {
        // append this filename
       
$message .= \"$uploaddir/$result['file_name']\r\n\";
    }
   
    // send email to client and to admin
    mail(\"
$adminemail; $user[email]\",
        \"Notice of File Deletion: 48 hours\",
        \"This file(s) will be deleted in 48 hours unless you change the\"
        .\" expiry information. If you do not own this file(s), you can contact\"
        .\" the administrator to have the information changed.\r\n\r\n
$message\",
        \"From:\");
}


// check file table for files belonging to deleted clients
$deleted_files = NULL;
$result = mysql_query(\"SELECT * FROM file WHERE id NOT IN (\".implode(', ', $clients).\")\");
while (
$file = mysql_fetch_array($result)) {
   
$deleted_files[] = $file['file_name'];
    unlink(\"
$upload_dir/$file[file_name]\");
}

if (count(
$deleted_files)) {
    // remove the db record for the unlinked files
    mysql_query(\"DELETE FROM file WHERE file_name IN (\".implode(\"', '\",
$deleted_files).\")\");
   
    // send email to admin
    mail(
$adminemail,
        \"NOTICE: files deleted from removed clients\",
        \"The following files were removed because they we're associated with\"
        .\" a non-existent client:\r\n\r\n\"
        .implode(\"\r\n\",
$deleted_files).\"\r\n\",
        \"From:\");
}


// check directory for files older than six months old
$oldest_date = mktime(0, 0, 0, date(\"m\")-6, date(\"d\"), date(\"Y\"));
$dir_obj     = dir($upload_dir);
while (
$item_name = $dir_obj->read()) {
   
$path = \"$upload_dir/$item_name\";
    if (is_file(
$path) && filectime($path)<$oldest_date)
        unlink(
$path);
}
$dir_obj->close();
?>

Mark Hensler
If there is no answer on Google, then there is no question.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Oh, brilliant, yes, of course we don't want to email people for stuff that we're going to delete now.

Thanks, Mark, Andy, for unravelling my knot, I'll apply this and test it and link to the final solution for other people to make use of, if needed.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

http://www.zerocattle.com/examples/expiredfiles.phps

That's my final solution and it works exactly as intended to work. What I decided to do about the expired client is to delete all associated files, then delete the client, not rely on the daily script to find stuff.

Many thanks, and I hope other people find it useful.

HOWEVER, because of the problem running PHP from crontab (use lynx, wget, or have PHP running as CGI), I'm going to probably rewrite it in Perl. Or, rather, I'm going to hope Wil says I only need to change a couple of lines, lol.

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

There's a problem running PHP from crontab?

Perl would be your best alternative. Could write a shell script, but you need interaction with a database.

Mark Hensler
If there is no answer on Google, then there is no question.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Yeah, if PHP is an Apache Module, you can't run it from crontab unless you use wget (which has to be available) or install Lynx and run it from the crontab.

The main problem is I need it to be very portable, to fit on many different server configurations, so I can't guarantee I'll have PHP as CGI, or have wget, and I pale at the idea of installing Lynx.

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

Well, every box that I've had access to has had both wget and lynx.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Either you're really lucky or I'm doing something really wrong. How do you tell if you have those things? I assumed I didn't because the crontab didn't work.

/bin/sh: /usr/local/lib/php/bin/php: No such file or directory and again, not working. :\

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

The include_path may not show the correct path to php. Check phpinfo() again and see if there is a --prefix in the "configure command"

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Configure Command

'./configure'
'--with-apxs=/usr/local/apache/bin/apxs'
'--with-xml'
'--enable-bcmath'
'--enable-calendar'
'--enable-ftp'
'--enable-magic-quotes'
'--with-mysql'
'--with-pear'
'--enable-sockets'
'--enable-track-vars'
'--enable-versioning'
'--with-zlib'
'

Does that tell you waht you need to know? What *I* need to know? lol.

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

There is no --prefix, so the default (/usr/local/php) would have been used.

So try:
/usr/local/php/bin/php -f /path/to/script.php

Or... better yet, try this in the shell first:
/usr/local/php/bin/php -r 'echo "found php!\n";'

Mark Hensler
If there is no answer on Google, then there is no question.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

I don't have shell access on my own server, and it's not working there. I'll try it on the client server and see if I can find it.

I'm getting very frustrated now, so I think maybe rewriting it in Perl would be easier. I say this BEFORE rewriting it, naturally.

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

You can fake shell access:

<?php
passthru
('sh /usr/local/php/bin/php -r \\'echo "found php!\n";\\'');
?>
'

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

okay, that didn't have any results, though it was really really cool to learn.

I went digging and found that my host uses MindTerm (java applet) to fake SSH, so I used that.

This is what it returned for a "whereis php"

/usr/bin/php
/usr/lib/php
/usr/local/bin/php
/usr/local/lib/php.ini
/usr/local/lib/php
/usr/include/php
'

So, now, which do you suppose is what I need?

I tried them all in the passthru above, to no avail.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Aha, okay, I have access to the server through shell. Of course, now I can't remember what the heck I needed to do this for. *argh* This over-a-few-days troubleshoooting is ridiculously hard. Sad

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

Whee, exploring again, found lynx. Woo!

lynx: /usr/bin/lynx /etc/lynx.cfg /usr/share/man/man1/lynx.1.gz
'

Okay, so I have lynx, and I have php somewhere I don't know how to find it. How do I use wget? I want to cover all my bases here.

Update: it WORKS! Finally. *phew* Using lynx is solid, does the job.

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

For wget, you basically just use: "wget URL"

There are quite a few options if you do "wget --help". Including no-cache, faking the User-Agent header, and faking the Referer header.

As for php, my guess is that /usr/bin/php is the binary.

Mark Hensler
If there is no answer on Google, then there is no question.

Suzanne's picture

She has: 5,507 posts

Joined: Feb 2000

okay, I'll check it out. Thanks, Mark. Smiling

Want to join the discussion? Create an account or log in if you already have one. Joining is fast, free and painless! We’ll even whisk you back here when you’ve finished.