Fragmented database access speed

[cytochrome]

Hi,

.................

@Francis: what do you mean? My C-drive is a normal hard-disk, only F: is the SSD and the other one's are external's, connected via USB 3.0 (H: and I: have USB 2.0, K: has USB 3.0, my laptop has only USB 3.0)

Gerd, it was a question about the Magician capabilities. I have right now my C: partition on an SSD that is too small (120 Go) and i want to change it for a bigger one (250 Go probably). The question is : how to migrate smoothly all my C drive (Win7 and programs like IM, DxO or Aftershot  to the new SSD. Will "Magician" be of any help in this migration?

@jch2103 : Thanks for the AnandTech links, its good reading before buying another SSD...

Francis

[jch2103]

I believe the software that ships with Samsung drives includes a clone program, although I don't remember if it's part of the Magician program. (I did fresh installs on the two computers in which I've installed Samsung drives.) There are also other clone programs available.

John

[Gerd]

Hi Francis,

as far as I can see, there is nothing in the Magician-software, that supports a migration.

I have earlier replaced my 600 GB with a 1TB harddisk. Here I connected the 1TB HD with a USB-case to my notebook and with AOMEI Partition Manager I transferred the 600 GB with a sector-by-sector mode. At the end I merged the 400 GB together with the 600 GB.
After replacing the disk I could boot and all was working! No problems with registrated software like PS, LR, MS-Office ...

I plan next year to change my 1 TB HD with a 1TB SSD.

[joel23]

Hi,

.................

@Francis: what do you mean? My C-drive is a normal hard-disk, only F: is the SSD and the other one's are external's, connected via USB 3.0 (H: and I: have USB 2.0, K: has USB 3.0, my laptop has only USB 3.0)

Gerd, it was a question about the Magician capabilities. I have right now my C: partition on an SSD that is too small (120 Go) and i want to change it for a bigger one (250 Go probably). The question is : how to migrate smoothly all my C drive (Win7 and programs like IM, DxO or Aftershot  to the new SSD. Will "Magician" be of any help in this migration?

Hi Francis,
I believe the trial version of Acronis True Image should be able to write an image (clone) of C: directly to an already connected new SSD. Afterwards you might need to correct/change drive letters.
Or use MiniTool Partition Wizzard which does it with three clicks. There are other Partition Tools around which will do as well.
After cloning you need to resize the new partition, 'cause it has the size of your old C:
In BIOS you might have to change the boot drive to the new one.   Hope I got all what is needed together now...

In all cases you need to check if the new partition is correct aligned, otherwise you will get huge performance problems up to 25 times slower than normal, especially with SSDs. 'cause for every 4Kb data (one cluster), two clusters have to be read when the partition is misaligned. For SSD it means, those two cluster first have to be deleted and than rewritten again. 
See attachment, hope it's to understand (above okay, below always half a cluster is used for the same data). W7 takes care about this, when a new partition is created. but I am not sure with cloning tools. Since it's 1:1 copy it should be okay, but better check.

You do this by launching 

Code Select Expand

wmic partition get BlockSize, StartingOffset, Name, Index in the command prompt.
BlockSize  Index  Name                   StartingOffset
512        0      Disk #6, Partition #0  512
512        0      Disk #2, Partition #0  1048576
512        0      Disk #4, Partition #0  1048576
512        0      Disk #1, Partition #0  1048576
512        1      Disk #1, Partition #1  105906176
512        0      Disk #0, Partition #0  1048576
512        0      Disk #5, Partition #0  1048576
512        0      Disk #3, Partition #0  1048576
512        0      Disk #7, Partition #0  1048576
512        0      Disk #8, Partition #0  1048576

Every number which is divisible by 4 in the starting offsets is okay. Here an older but still valid article about that Had to stripe the "still valid", because it claims that a misaligned partition need to be reformatted to align it. But after four years (the article is from 2010) there are tools which are able to realign misaligned partition without reformatting them. Thanks god.

[attachment deleted by admin]

[cytochrome]

Thanks Gerd and Joerg, it is planned for next month. I will think thoroughly about all this, I have a lot of stuff and settings I would not like to loose.

Last year while looking for migration tools (to move from my old Dell to my new PC) I came across XXClone. I didn't use it finally, managed everything "by hand" but the concept seemed OK. Has anyone used it?

Francis

[JohnZeman]

Last year while looking for migration tools (to move from my old Dell to my new PC) I came across XXClone. I didn't use it finally, managed everything "by hand" but the concept seemed OK. Has anyone used it?

A few years ago I used XXCLONE for awhile but later changed to the free Acronis cloning tools for cloning specific hard drive makes (Seagate and Western Digital are the drives I tend to use).  I clone my C: drive to a new spare at least once a month just so I'll have it for a backup should the C: drive suddenly go bad and that does happen once in awhile.

XXCLONE was created by the author of xxcopy, a program I still use today for general incremental backups of my D: drive.  however I'm starting to think about changing to something else for my incremental backups because I know the author of xxcopy and XXCLONE is about ready to retire.  I'm old as dirt and he's even older than I am. lol

[joel23]

Thanks Gerd and Joerg, it is planned for next month. I will think thoroughly about all this, I have a lot of stuff and settings I would not like to loose.

As said with a clone you get a 1:1 copy of the partition. Nothing would get lost.

In the worse (if the change from one "C: to the other C:" does not work for you on the first attempt) you always have the "old" SSD.
Also make sure you have a boot CD (e.g. made by one of these Partition Tools or Hiren's Boot CD which btw. boots a minimized XP from which you can launch the mentioned "MiniTool Partition Wizzard") by which you are able to switch back drive letters and the active partition, just in case.

[Gerd]

Hi,
I do not prefer incremental backups, if one part is wrong, all is wrong ...

I do a complete file-backup (automatic in the background) to my 4GB-HD and also a backup by copying changed/new files from selected directories (incl. sub-dirs) to another HD via AllwaysSync.

With this way I have allways a readable safety-copy (hopefully) ...

[joel23]

Hi,

@Jörg: why shell I disable what delivers the speed? I want to have the max. speed and it doesn't matter, if extra RAM is used or not ...

Hi Gerd,
this was meant for getting SSDs real speed, as you did later.

Nevertheless, AFAIK even Samsung does not mention speeds above 1.200MB/s for Rapid mode, so you're lucky to beat their specifications...

but a SSD works also like a RAM-disk, or ...

Almost. A SSD is far slower as RAM and aside this there is a (small but still existing) chance of losing data when you encounter a sudden power failure, since RAM is volatile.
I just don't believe this will actually help much. What does you make believe it gives you more speed? Looking at the numbers?
Maybe when saving files larger than 1 or 2 GB by an application. But when copying, moving files or writing metadata to images, the rather low performance of the other HDDs determine the speed.
To get a clue, what means large excel files? >400MB?

...the other one's are external's, connected via USB 3.0 (H: and I: have USB 2.0, K: has USB 3.0, my laptop has only USB 3.0)

I am not a fan of connecting otherwise pretty fast disks via USB - but K:, your 4TB HDD is pretty lame for an USB-III connection; I would expect 80-90MB/s if not more.
Is there an eSATA or combined eSATA/USB port on your laptop?

[Gerd]

Hi Jörg,

this is OT,what I have to do with Excel, but I will explain short:

We (the company I work for) are using Excel as flexible database. We need to analyse complex combinations (connecting the results of several pivots). The basic db has nearly 200 columns and can have up to 900.000 rows. If you have to fill such long column via index&match with data from another big sheet, then it takes nearly 3h. The CPU is used by 100%, the van is sterting with a higher speed and the temp inside is raising from 45 degree to 80 degree. A pivot combines this amount of rows to 50.000 products and for special analytics I need the each available price for all products in columns to the right (via a pivot). This can create several thousands columns ... such Excel-sheet can be 480MB ...  usually my Excel-sheets are between 5 and 100 MB.

Back to the topic:
One of my default background programs is SmatDefrag (IObit). This prog. defragments my C-drive (and only this) in periods, where my notebook is not so busy.

I was a little bit astonished, that my USB 3.0 drives are not much quicker than my older USB 2.0 drives, all are connected to USB 3.0 ... Oooops .. here I think, I can improve!
They are first connected to a van-stand and then to my notebook. This can be a bottleneck, I will look for a newer USB-hub and test again. I made this speed-up experience with myUSB-cardreader, the older one (USB 2.0) need 2,5 sec. per pic and the newer one (also USB 2.0) needs 0,5 sec. per pic.

I'll keep you informed.

[joel23]

this is OT

the last 2/3 of this thread went OT. But I believe this is not that bad.

...such Excel-sheet can be 480MB ...  usually my Excel-sheets are between 5 and 100 MB.

I was just asking about the size of the files, because either should be saved by 1-2sec (I included a bit for latency) or less to your SSD, even when RM is disabled.
FYI: the 840 Evo has a read performance bug - the bug fixed is announced for today. So watch out.

They are first connected to a van-stand

Not sure what this is, but if it's the USB-Hub, it of course have to be USB-III as well.

I'll keep you informed.

Okay, thanks. Hope a new Hub will help. Since you use it for backups, they must take ages

[joel23]

With this way I have allways a readable safety-copy (hopefully) ...

Hopefully. How many generations you keep?

[Gerd]

Hi,

thanks for the info about the EVO-bug, I've set it in my to-do-list!

I keep 2 generations as file-backup (old-one and actual one).

Here my new results with an USB-3.0-hub (right pics shows results with USB 3.0 hub ...yes, it speeds-up):


[attachment deleted by admin]

[Richard]

the last 2/3 of this thread went OT. But I believe this is not that bad.

I agree that most of this thread is Off Topic but I hope it is information that will benefit IMatch users.

[joel23]

I keep 2 generations as file-backup (old-one and actual one).

So when you encounter an error (corrupt images, excel sheets, etc.) applied three days ago, you're lost. Acronis btw is very reliable in making incremental backups of whole disk images.

Here my new results with an USB-3.0-hub (right pics shows results with USB 3.0 hub ...yes, it speeds-up):

Wow, you're quick.
Three+ times the former speed and even the other disks participated a bit by it, as I almost expected.

[Gerd]

Hi,

a final feeling: today I have installed the new firmware for my 840 EVO ssd, later the 29 MS updates ... now all is running much faster: LR with nearly no waiting time, to load a pic in max. resolution, less than 1 sec., also IM is running like the devil ... opening my catalogue with nearly 130.00 pics is done in 5 sec. and then 15 sec. to get the first pic in the viewer ... scrolling reacts immediately .... all in all: an amazing performance ... 

[joel23]

Thanks Gerd and Joerg, it is planned for next month. I will think thoroughly about all this, I have a lot of stuff and settings I would not like to loose.

Last year while looking for migration tools (to move from my old Dell to my new PC) I came across XXClone. I didn't use it finally, managed everything "by hand" but the concept seemed OK. Has anyone used it?

Francis

FYI: Samsung has a data migration tool

[cytochrome]

Thanks to all, lots of info and ideas on how to migrate properly. I have an USB3 HD dock that accepts 2.5" drives so should be easy to set up.

I went back to the XXCopy help, it is interesting in that it copies files and folders, not disk sectors, so does un-fragment at the same time (see Richard, I am back at the OT  ). But I'll probably go to a more classical cloning tool.

I have Paragon Drive Copy, never used it yet but often used Paragon Partition Manager to expand or merge partitions, never failed so i have some confidence. I will also test some of the other programs cited. This forum is a real info mine.

@Joerg, I had seen the Samsung migration tool on their site. They insist it is for Samsung drives only. I will probably buy one but my present SSD is a Kingston. Not sure how all this will cooperate. Anyway thanks again and sorry for derailing the threat...

Francis

[RalfC]

Actually, if you dare to use some non-Windows tools  (bootable from CD or USB), Clonezilla http://clonezilla.org/ for cloning and restoring and GPartEd http://gparted.org/ for changing partion sizes might be worth a look.

Regards,
Ralf

[Gerd]

Hi Francis,

when I planned my migration, I was also thinking about copying files and folders, but then my idea was, to get an exact copy of my system and data with the hope, that I do not need to install it all new ... and it works ... but that was my old pc. My new pc was delivered with an installed Win8 and here I need to copy files and folders, selectiv. It took much more time, than the migration of my old notebook.

If you plan to do it by file and folders, you should start with a clean new Win-installation and then re-install your extra programs and then fill-up with copying specific files and folders.

just my 2 cents ...

[cytochrome]

@Ralf_C: on my previous PC I had a Linux partition, and I still have an Ubuntu bootable DVD that I use if for some reason I can't delete or rename a file in Windows. But I am not so familiar with Gparted so...

@Gerd Thanks for the comments. I did it just like you when I went from old to new PC, but now that I have a PC that works well, with programs installed and organized at my liking I wish to conserve this when going from a too small SSD to a bigger one. If you read the XXClone stuff you will see that it will copy files and folders AND prepare the new drive to be bootable. In principle no need to re-install anything.

But I think I'll go with a bit to bit cloning solution. I will take my time to think about and do some preliminary experiments. The only benefit of age is that I get more cautious (and less entrepreneurial which is sad).

Francis

[RalfC]

@Ralf_C: on my previous PC I had a Linux partition, and I still have an Ubuntu bootable DVD that I use if for some reason I can't delete or rename a file in Windows. But I am not so familiar with Gparted so...

OK, you convinced me to write a bit more about Clonezilla and Gparted. 

Both exists as so-called Live-CD / Live-USB, i.e. it is possible to boot those systems directly and no installation needed.

Clonezilla can be used to save and restore disk-images (or partitions) including the partition table, etc. onto an external harddisk. This can be used to create a backup of the current state of the disk and the restore will bring the system back into use. (Like with other disk imaging programs). Clonzilla is fully based on the text based UI, but it leads through the process by offering possible choices.

Gparted is a partitioning tool, which is able to create, delete and resize (increase and decrease) partitions and it is also able to move(!) partitions for almost any filesystem (encrypted systems excluded). It has a graphical UI.

In my experience, the migration tools, from Kingston but also the commercial tool from Paragon, are not able to migrate a harddisk to a smaller SSD even if the used space is below of the SSD's capacity and/or might have other restrictions. [I do not know the Samsung tool, but in the user manual is mentioned that the OEM recovery partition is not cloned and having several OS on the harddisk might result in a nonfunctioning setup].

In those cases the above mentioned tools can be useful:

• Backup the Harddisk (Clonezilla) [A good backup is never wrong  ]
• If needed: Resize and move the partitions to the target size of the SSD (Gparted). Be cautious that the used space is not bigger than the SSD size
• Transfer the image from HD to SSD including the partition table etc. (Clonezilla)
• Make some final adjustments for the partition(s) on the SSD (e.g. GParted)

Regards,
Ralf

[cytochrome]

Well Ralf, you are taking me by the hand!! I will make a Clonezilla DVD and try once I bought a second SSD.  Do I understand well that you recommend to clone first my actual ssd to an rotating HDD, and then from there to the new SD. why not directly to the new one? I will mount it on an USB3 dock, should work, non? If I buy a 250 Go SSD it will have only one partition: C: so my data that is stuffing my present 120 Go SSD will be OK.

I don't understand the comment about cloning to a smaller disk, I will of course clone from a smaller to a bigger one. I mentioned XXclone because their technique to get an un-fragmented disk from a fragmented one is tempting.

Francis

[RalfC]

Well Ralf, you are taking me by the hand!!

I thought, I describe the generell process for the benefit of all. 

I will make a Clonezilla DVD and try once I bought a second SSD.  Do I understand well that you recommend to clone first my actual ssd to an rotating HDD, and then from there to the new SD. why not directly to the new one? I will mount it on an USB3 dock, should work, non? If I buy a 250 Go SSD it will have only one partition: C: so my data that is stuffing my present 120 Go SSD will be OK.

I would make a backup image to a normal harddisk (good practice, anyhow). Clonezilla can also copy from device to device directly, but I never tried it.

I don't understand the comment about cloning to a smaller disk, I will of course clone from a smaller to a bigger one. I mentioned XXclone because their technique to get an un-fragmented disk from a fragmented one is tempting.

Your case is a simple case for cloning/copying when the target disk is bigger than the old one (in your case 120 GB to 250 GB).

Let me give an example for cloning to a smaller disk:
The computer has a harddisk with 500GB and that shall be replaced by an SSD with 250GB and the harddisk has 3 (or more) partitions:
1) Windows Boot (size 500MB); 2) C: (size 460 GB, 200GB used); 3) OEM recovery (size 30GB, at the end of the disk) [Windows shows only partition C: !]

If you want to move such a setup to an SSD, you'll have to make the C: partition smaller and move the OEM recovery partition.
This is not easily possible with Windows tools: Windows stores some required. unmovable  data in the middle of the partition, i.e. Windows can not make the partition small enough. In addition, Windows can not move a partition. Gparted is able to decrease the size to the required value (because Windows is not running and locking its files) and also move a partition.

Regards,
Ralf

[cytochrome]

Thanks for the clarification Ralf.

Francis

[khfnet]

Hi,

This post will appear permanently under "show new replies to your post".
But it is always unchanged.

Why?

[cytochrome]

I bring this thread back to life because it gave the push to change my Kingston SSD that was cramped (120 Go, 90% used) for a 250 Go Samsung EVO 840 where Windows and IMatch are more at ease.

To clone the Kingston, after reading again all the posts and advice here I used one of the programs suggested by Jörg: Minitool Patition Wizard because I am really more at ease in the Win world than with Linux solutions. MPW is free for private users, simple to use, options are few and clear, it takes 3 clicks to clone a disk and it works.

It took a bit of time to get the Samsung recognized, really my fault cause I had not read the Asus Bios instructions. When I got it right everything including IMatch was there.

Are there benefits? There should be, in the CrystalDiskMark test it runs much faster than the Kingston but in real life with Imatch it is not that faster.  As always it is the slowest parts in the process that set the overall speed. The image files are on a 7200 rpm hdd and although IMatch runs on the Samsung I believe it is Exiftool that is the bottle neck and puts a brake on IM.

But IM starts faster, the delay to open the DB and open the files windows is shorter, white screens are transient (some seconds), and no hang (but I had very few with system and IM on the Kingston).

Francis

[attachment deleted by admin]

[P.Jones]

I bring this thread back to life because it gave the push to change my Kingston SSD that was cramped (120 Go, 90% used) for a 250 Go Samsung EVO 840 where Windows and IMatch are more at ease.

To clone the Kingston, after reading again all the posts and advice here I used one of the programs suggested by Jörg: Minitool Patition Wizard because I am really more at ease in the Win world than with Linux solutions. MPW is free for private users, simple to use, options are few and clear, it takes 3 clicks to clone a disk and it works.

It took a bit of time to get the Samsung recognized, really my fault cause I had not read the Asus Bios instructions. When I got it right everything including IMatch was there.

Are there benefits? There should be, in the CrystalDiskMark test it runs much faster than the Kingston but in real life with Imatch it is not that faster.  As always it is the slowest parts in the process that set the overall speed. The image files are on a 7200 rpm hdd and although IMatch runs on the Samsung I believe it is Exiftool that is the bottle neck and puts a brake on IM.

But IM starts faster, the delay to open the DB and open the files windows is shorter, white screens are transient (some seconds), and no hang (but I had very few with system and IM on the Kingston).

Francis

Regarding the Samsung EVO drive this may help

http://www.extremetech.com/computing/192419-samsungs-840-evo-repair-tool-available-to-download-but-be-careful-of-the-caveats-when-you-update

http://www.samsung.com/global/business/semiconductor/minisite/SSD/global/html/support/downloads.html

[cytochrome]

Thank you for the info. I have not seen any problem (yet) but the drive is only active since 2 days. It came with a software CD with Magician and a migration tool. I did not use it because I read several posts about problems. So I used Minitools' Partition Wizard which is fine and foolproof (just what I needed .

The programs on the CD date all back to 2013, I will download the last versions and also the EVO 840 Repair tool in case of problems (I still have the Kingston SSD if some silliness appears).

I bought the drive from Amazon, they have little stock so may be it is a already a fixed issue on this one. We will see...

Francis