Full Time
to be discussed
TBD
Jun 25, 2013
here is a trail task:
we have a system consisting of a:
- database
- website
- search spider ( a script)
Spiders (which are running in parallel on the same server) are downloading info from html pages with the use of (HttpWebRequest http://msdn.microsoft.com/ru-ru/library/system.net.httpwebrequest.aspx) and put the parsed data into MSSQL database.
As far as they run in paralel, they load MSSQL with overhead requests.
Website display the data using Full Text Search (MSSQL), which is also loading the database with requests.
Result: not optimal MSSQL requests load.
Task is to change the spider for the data to be placed first into an intermediary file, not right to the database.
And than to write the code to move the data from the intermediary file to a database only by a single thread, thus easing the MSSQL load.
We need you to redesign the following classes:
\Skytender\Spider\Utils\DataStorage\DBStorage.cs
\Skytender\Spider\Utils\DataStorage\IRecordStorage.cs
At present time DBStorage.cs is called from spiders code to save the data right to the databse, you have to:
1. write one more calss with IRecordStorage to record data into intermediary file on disk.
2. Write a separate code(standalone program), that will import the content of the intermediary file to a MSSQL database.
Attached:
sourceode ( .zipped must be changed to .zip): https://docs.google.com/file/d/0B-7KKuRDlT5zcF81RGRRZGE5Vk0/edit?usp=sharing
MSSQL database backup(restore through MSSQL management studio): https://docs.google.com/file/d/0B-7KKuRDlT5zOXcyeHpFQmR4ZEE/edit?usp=sharing
We have serious plan for a web development of the service, which serves many subscribers in Russia. So we need a long term ASp.net coder with a great expirience in ASP.net, XML, crypto service knowlidge is great plus.