Custom Search
Hey everyone,
I've been in the process of figuring out how to implement searching on my new project, and I'd like to get some input.
The qualifications:
-the data is only accessible by HTTP through a valid login
-so Google's custom search won't work
-search with different options: posts, users, companies, and comments
My idea:
-have a CRON job that runs 4x a day, evenly spaced, that basically indexes each applicable MySQL data into a .txt file (CSV formatted)
-when someone searches, they select where to search
-depending on the selection, the appropriate .txt file is opened (read only), and the information is 'looked for'...returns certain pieces of data upon results
Is this going to work? Is this going to be efficient?
I already wrote the scripts that do the 'MySQL indexing', and those run pretty quickly (also implements locking so that, for whatever reason, it can't run two at the same time and screw everything up).
I've thought about using some pre-existing solutions, but I was reading up on a few, and I noticed they used indexing, while storing in text files...I thought I should just write my own to get exactly what I want.
I'd appreciate any comments, criticisms, or suggestions...
Thanks
pr0gr4mm3r posted this at 01:42 — 28th April 2007.
He has: 1,502 posts
Joined: Sep 2006
What exactly is the search for. I don't know why you would want to save the database in a text file when a database is made to be queried with criteria.
kb posted this at 03:09 — 28th April 2007.
He has: 1,380 posts
Joined: Feb 2002
Well, I'm trying to minimize the database queries. I don't have access to memcache, so I have to watch database connections and things...I'm building a product made for large-scale usage.
The theory, at least in my mind, is that by access the entire database 4x a day, and the searching through that...it will actually make load times faster, because it doesn't have to have yet another mysql query being processed.
Does that make sense?
pr0gr4mm3r posted this at 03:33 — 28th April 2007.
He has: 1,502 posts
Joined: Sep 2006
I'm not sure that makes sense. A database is made to be searched through. A text file is not.
How large scale is this going to be? I run queries on tables that are 100MB in size of text with no problem.
kb posted this at 04:02 — 28th April 2007.
He has: 1,380 posts
Joined: Feb 2002
Well, I guess the scale depends on the use...but it's intended to be massively big.
Is there a native way to run a search function (not a SELECT, but an actual ...search), within MySQL? That would save resources...
sherzodr posted this at 18:57 — 1st May 2007.
He has: 6 posts
Joined: May 2007
Yes, it is possible. You can use any of those free indexing tools. Check out swish-e! Using swish-e you can create an index of pretty much any data, be it in databases, static pages - you name it.
QVote.net - Web Polls made easy and affordable
kb posted this at 20:38 — 1st May 2007.
He has: 1,380 posts
Joined: Feb 2002
Yeah, but the whole point is that I want to write something myself, so not only do I learn something, I also get to have it do exactly what I want, and not waste any unnecessary resources.
Want to join the discussion? Create an account or log in if you already have one. Joining is fast, free and painless! We’ll even whisk you back here when you’ve finished.