Should I use Flatfile or MySQL

He has: 688 posts

Joined: Feb 2001

----------------------------------
New situation near bottom of this
thread, so ignore this top area.
----------------------------------

I've got some databases that have been maintained for years in MS Word. I then "save as HTML" and upload them to my site. Here's one of the five "databases":

pstvalumni.com/dir00a.shtml

They're not sortable or searchable, and it takes a half dozen steps to update. So I'm thinking of converting it to a real database.

My question is whether I should use a mySQL database, or whether my database needs doesn't justify anything more complicated then a flatfile? I have a total of about 200 "rows" (people) and somewhere between 8 and 14 "columns" (data for each person), depending on whether hyperlinks for fields need a separate field. So I guess I'm looking at about 2,000 fields? Will that bog down a flatfile? And even if I use mySQL, won't I only be making only one table within the database?

I'm a newbie to databases and I'll probably ask how to set it up later, but for now I just want some advice on which direction I should be heading for my particular needs.

Thanks Smiling

Busy's picture

He has: 6,151 posts

Joined: May 2001

Sounds like you already know how to set up and use a flatfile so maybe using a database (mysql) would be better off in the knowledge area. no matter how simple it is, its great experience for when you need to do a larger job.

My pick would be database as you could make it searchable, sortable and easy to update/change/add/delete etc

Mark Hensler's picture

He has: 4,048 posts

Joined: Aug 2000

I'm not a fan of flat files simply because of the server load it produces when under heavy use.

Being that the content is static, I would recommend storing the data which ever way is easiest for you to code. Then, create a mechanism to spit out static pages whenever a change is made to the DB. This will reduce server load, increase response time, and be a really cool project. 8]

Mark Hensler
If there is no answer on Google, then there is no question.

Peter J. Boettcher's picture

They have: 812 posts

Joined: Feb 2000

Even though you don't think you'll need it now, save yourself future headaches and go with a real database like mySQL.

That way you won't be hampered by any changes you may want to make in the future, like adding the ability to store more data. By having the data in a database it also allows you to use the data for a myriad of different applications.

PJ | Are we there yet?
pjboettcher.com

They have: 112 posts

Joined: Apr 1999

By using flat files, you will need to create your own functions to do string manipulation, sorting, database locking etc, which is not worth it. My suggestin is go with MySQL.

Joe
Webhosty.com

He has: 688 posts

Joined: Feb 2001

Okay I'm lost. Please help me. Sad

I've "converted" my "database" to an excel file (Excel 97 SR-2). I've tried over and over again to convert it to a mySQL database by saving the excel file as some other format and then trying to import it into a database using phpMyAdmin. Everytime it says error in line 1.

Throw away everything I tried and start from scratch. How do I convert my .xls file into a mySQL database (phpMyAdmin is available to me)? My .xls file is 176 rows by 10 colums. I assume that would end up being one table inside a mySQL database, but I don't know anything about this so I'm just assuming.

Help? Thanks!!!

Peter J. Boettcher's picture

They have: 812 posts

Joined: Feb 2000

What format did you save it as in Excel? Tab-delimited text?

You shouldn't have any problems importing that text file into mySQL. Maybe try bypassing phpMyAdmin, see this link for more info: http://www.mysql.com/doc/en/mysqlimport.html

PJ | Are we there yet?
pjboettcher.com

He has: 688 posts

Joined: Feb 2001

The info in the link looks good but I don't understand enough of the basics to know what it means. The problem is that I don't even know where to start Confused I'm completely new to mySQL. I guess I should begin with what I was doing, then maybe somebody could shout out at me that I'm going about it all wrong.

I tried many formats, all of which say "You have an error in your SQL syntax near xxxx at line 1". I see in the phpMyAdmin that it says "Or Location of the textfile." so I've been concentrating on exporting the .xls file as a text file.

I created a database first and tried to "import" my data but it didn't work. So I figured that maybe I need to create a table first, so I created a table with 10 fields and tried to import using the same method ("Or Location of the textfile.") but that didn't help.

He has: 688 posts

Joined: Feb 2001

Hey, I made some progress! After I created a table within the database, I used the link "Insert data from a textfile into table" which did not give me errors. I said that it successfully entered 176 rows, which is the correct number of rows in my text file.

However when I then hit "browse" it shows me all of each row's column data showing up in the first field of the table Sad

Busy's picture

He has: 6,151 posts

Joined: May 2001

my first time i had the rows but no content at all.

check your quotes and commas, are you missing any and lookout for things like: "Hi, I'm Bob" should be "Hi, I\'m Bob" .. very easy oversight to make.
Also were the contents in the text file correct with db tables?, if you declare notnull and upload/insert as null(empty) you'll get out of align. hope this hasnt confused you.

an example of what i use:
use dbname;

insert into dbname values
(null, "n is after m", "", "318", null),
....

etc etc, dbname being your database name, the first null is the auto increment number, n is after m is just some rubbish i put there Smiling but its the content, the one after is a blank field which coulc or could not contain data, but if i missed this out the next number would move over one and would be one short so throws the whole thing out of whack.
[note]Hope his make sense, still on first coffee of the morning[/note]

They have: 447 posts

Joined: Oct 1999

why not drop the import as file idea, save the files as csv in excel, and do something like this:

<?php
$fp
= fopen('datafile.csv', 'r');
while(
$coldata = fgetcsv($fp, 10000)) {
   
$thequery = \"INSERT INTO mytable VALUES('$coldata[0]','$coldata[1]','$coldata[2]','$coldata[3]')\";
    mysql_query(
$thequery);
}
?>

He has: 688 posts

Joined: Feb 2001

Sorry for my novice-ness. Your suggestion sounds promising but what do I do with it? (other than changing datafile.cvs to whatever I named it) Is this code to place in a php page? Or is this some command code to use within phpMyAdmin? And what's that coldata stuff?

Sorry but nearly all of my knowledge is in simple .shtml stuff. Maybe this project is a little beyond my current capabilities Confused

Busy's picture

He has: 6,151 posts

Joined: May 2001

the code Rob showed is included in a .php page

phpMyAdmin is real easy to use, but as a wise Mod here mentioned (Mairving) it's better to learn the web page method, using INSERT, UPDATE, ...
At first it all looks hard but do it a couple of times and its quite easy. then once you know the basics of written MYSQL then go back to phpMyAdmin
Kevin Yank has a few tutorials around the place, even on mysql.com itself

He has: 688 posts

Joined: Feb 2001

I just wanted to update everyone who tried to help me. I finally got my data into a mySQL table. I discovered the difference between my cvs output and a sample dump file was that the dump file had spaces following all the commas, while my cvs file did not. I just ran a find and replace all, for all commas to a comma with a space after it. I then saved it and changed the file extention from .cvs to .txt. It was a back-@ss way to do it but it worked!

Smiling

Peter J. Boettcher's picture

They have: 812 posts

Joined: Feb 2000

Way to go Mike! Doesn't really matter how you beat a problem, as long as you beat it.

Busy's picture

He has: 6,151 posts

Joined: May 2001

congrats, and thanks for sharing the outcome

Want to join the discussion? Create an account or log in if you already have one. Joining is fast, free and painless! We’ll even whisk you back here when you’ve finished.