<$2000 Computer to handle mass data work

sepefrio

Honorable
Nov 4, 2013
4
0
10,510
First, thanks in advance for any advice I may receive. I have sold my company on a new business intelligence tool for use on one of our projects. They bought me the software, but not any hardware (yet). The BI software is nothing special computer needs wise, but the amount of data I need to first scrub then run through it is very large (and getting larger). I currently have to take source data from almost a dozen different sources, of course in a dozen different formats, and turn it into one large file in a specified format. I've written some middle-ware DB's to do this work for me. Each file of source data is manageable at only a few 100,000 records each (per day). But when I put them all together, the work computer I have (old Dell Latitude laptop) just coughs and chokes. In fact, I have actually brought my iMac to work just to handle the load for now.

Well, I want to take my iMac home and return it to personal use but I also want a new rig built just for the above purpose. I know, the company should buy it for me, but we all know, IT budgets are limited and what we think we need to get the job done isn't the same as what they think you need. This project is still in proof of concept and if it is accepted, I will see a real nice boost in pay. As such, I'd rather depend on something I built instead of being handicapped by something that might be just good enough.

From what I know hardware wise, to accomplish this task, the chip and amount of RAM is key. There is no audio or video involved and I could care less about them anyways. Hard drive space I assume is also a key item. I already have a monitor so I I just need a rig. I'd like to stay under $2000, but if there is a huge benefit, I will consider higher priced outfits.

Again, thanks in advance and I hope to hear something soon. I'd like to make the purchase within the next week so I can move forward on the project. Thanks!
 
Not sure what open cl or cuda is.

I receive the data in excel, flush through Access and have Tableau (Tableausoftware.com) read the info from Access (may switch to SQL Server after I learn it).
 
basicy you can designate a portion of the ram in this case i recomend 32gb ot house your data, ram reads EXTREMELY fast on the order of the equilven of 6-8 top teir ssd in raid 0 in a best case senrior however its CRITICAL that you either have very rare power lose or better yet a ups because ram requires power to mantain data in its memory so if it powers down before it can be writen all data is lost
 
Yup, just did some reading on that, the two drawbacks are startup and shutdown as it takes some time to write the data between the RAM and disk. But I really like the idea and yes, with an UPS, the lost power problem is very minimal at best. Thanks again!