already published on 1/1/2019.
The files are all in the folder for ODLK1 result: https://drive.google.com/drive/u/0/folders/1wIf97_bLDvnb1WalOsjU9vODZqRZ_WNw (it should be open to everyone)

Very good. Thanks.

I'll see the results.

https://boinc.multi-pool.info/latinsquares/forum_thread.php?id=57&postid=656

Natalia Makarova
So we have 226322 +184328 = 410650 CFs.

I got exactly that many (410650) Mariažne CF DLK with my postprocessing program. Looks like I did not make a mistake in the program. (batch Sep 2018)
From them:

Found Fancy CF:
count[1]: 409774
count[2]: 876
All: 410650
Found CF co-squares: 403590

But I did not perform check of uniqueness agains the 5M database. Only (implicit) check withing the file itself.

What form is your database? Is it just a text file? With that many entries it should be a real database, like sqlite, postgress.
If it is just a file, program to check for uniqueness is very easy to make.

I got exactly that many (410650) Mariažne CF DLK with my postprocessing program. Looks like I did not make a mistake in the program. (batch Sep 2018)
From them:

Found Fancy CF:
count[1]: 409774
count[2]: 876
All: 410650
Found CF co-squares: 403590

But I did not perform check of uniqueness agains the 5M database. Only (implicit) check withing the file itself.

What form is your database? Is it just a text file? With that many entries it should be a real database, like sqlite, postgress.
If it is just a file, program to check for uniqueness is very easy to make.

Tomas Brada
Fine!
I will talk about the database a little later.

Unfortunately, our database consists of separate parts.
Now we will consider the parts of the database that are related to the ODLK1 project.
I will give all links to these parts of the database.

Then you need to combine these parts and check for uniqueness.

Check for uniqueness is a simple procedure.
But it is necessary to take into account that parts of the database are published in different formats (format 1 and format 2).
If you perform a check for uniqueness using the program of Belyshev kanonizator_dlk_1.03, then different formats do not matter.

I give all parts of the database that are related to the ODLK1 project.
Now we have to combine these parts and check the resulting database for uniqueness.

1. January 16, 2018 posted the first part of the database, containing 500,000 unique CF ODLS
https://yadi.sk/d/Of8_1d-b3RW43F

2. The second part of the database containing 500,000 unique CF ODLS is laid out.
You can copy this DB from the following link
https://cloud.mail.ru/public/FLXU/DgejKWpVc

3. The third part of the database (500,000 unique CF ODLS)
https://cloud.mail.ru/public/6Waa/GikBSgP9o

4.The fourth part of the database (500,000 unique CF ODLS)
https://cloud.mail.ru/public/8tqZ/DYbt9iCqN

5. Results of the ODLK1 BOINC-project for May 2018

447612 unique CF ODLS

©2018 Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/LeB8/8BVqTzBfj

Created on June 24, 2018

6. Results of the ODLK1 BOINC-project for June 2018

486061 unique CF ODLS

©2018 Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/ENBK/Ljqnqji79

Created on July 2, 2018

7. Results of the ODLK1 BOINC-project for July 2018

407150 unique CF ODLS

©2018 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/56UQ/sNubiDF71

Created on August 2, 2018

8. Results of the ODLK1 BOINC-project for August 2018

400553 unique CF ODLS

©2018 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/92Zh/MwrNAcdW9

Created on September 3, 2018

9. Adding to the database in April 2018

9294 unique CF ODLS.

©2018 (C) Progger
©2018 Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/L68Y/fJZkA3mYf

Created on October 5, 2018

10. Addition to the database from the manual project

67879 unique CF ODLS

May 1 - November 4, 2018

©2018 (C) Natalia Makarova & Demis & ice00
©2018 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/EAbb/fmt9Pg1RV

Created on November 5, 2018

11. Results of the ODLK1 BOINC-project for September 2018

407995 unique CFs ODLS

©2019 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/LcCy/rERVzo8qv

Results of the ODLK1 BOINC-project for October 2018

404478 unique CFs ODLS

©2019 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/oPwi/fv9FetwmS

12. Results of the ODLK1 BOINC-project for November 2018

355634 unique CFs ODLS

©2019 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/G1Q2/Qy343sBQg


Results of the ODLK1 BOINC-project for December 2018

343240 unique CFs ODLS

©2019 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/GTHq/qv6ZXcn4R

I can check the arithmetic union of parts of the database.

I repeat: the uniqueness check can be performed in different ways.
But with other methods it is necessary to take into account the format of the results.

If you use the program of Belyshev kanonizator_dlk_1.03, the format does not matter.

What is the DLS format?

1. Format 1
in DLS the first row is normalized (0 1 2 3 4 5 6 7 8 9).

Example

0 1 2 3 4 5 6 7 8 9
1 2 0 4 3 6 5 9 7 8
2 0 1 5 6 4 3 8 9 7
3 6 7 8 2 9 1 0 5 4
8 5 4 7 9 2 0 6 1 3
5 4 8 9 0 7 2 1 3 6
9 8 3 6 7 0 4 5 2 1
6 9 5 0 1 8 7 3 4 2
7 3 9 2 5 1 8 4 6 0
4 7 6 1 8 3 9 2 0 5

2. Format 2
in DLS normalized main diagonal (0 1 2 3 4 5 6 7 8 9).

Example

0 2 3 4 5 9 7 8 6 1
2 1 7 5 9 3 8 6 0 4
8 4 2 1 0 6 5 3 9 7
6 9 0 3 8 7 4 2 1 5
1 6 8 0 4 2 9 5 7 3
9 3 4 7 6 5 1 0 2 8
4 7 1 8 3 0 6 9 5 2
5 0 9 2 1 8 3 7 4 6
3 5 6 9 7 4 2 1 8 0
7 8 5 6 2 1 0 4 3 9

8. Results of the ODLK BOINC-project for August 2018

45864 unique CF ODLS

©2018 (C) Progger

Link
https://cloud.mail.ru/public/BQw7/CQaxtWkQF

This is mistake!
These are the results of the ODLK project.
I'll fix it now.

I fixed the mistake here
https://boinc.multi-pool.info/latinsquares/forum_thread.php?id=54&postid=664#664

I will now check the arithmetic union of the parts of the database.

Arithmetic union of DB parts contains 5,329,896 CFs ODLS.

Are all these CFs unique?
When manually checking the uniqueness for individual parts of the database, I could be mistaken.

ice00
I hope you can now process the results, since the software was created for Linux system by Tomas Brada.

Arithmetic union of DB parts contains 5,329,896 CFs ODLS.

I've got the same number of squares: 5'329'896, but that is not union, just concatenation.
However, kanonizator_dlk found only: 5'329'884 CF DLK, that is 12 difference.
Program kanonizator_lk reports that is can only read 5329884 squares and found only 5212580 KF LK.
My program reports only 10 duplicate squares (disregarding format).

We need to check where the inconsistency comes from.

However, kanonizator_dlk found only: 5'329'884 CF DLK, that is 12 difference.

It is quite possible.

Program kanonizator_lk reports that is can only read 5329884 squares and found only 5212580 KF LK.

CFs LS may be less than CFs DLS. It is right.

My program reports only 10 duplicate squares (disregarding format).

Can you identify duplicates?
Can you figure out which part of the database contains duplicates?

Thank you very much for your work!
Now we have a common database for the ODLK1 project.
This database contains 5'329'884 unique CFs ODLS.
Fine!

I assume that duplicates are contained in this part of the database

9. Adding to the database in April 2018

9294 unique CF ODLS.

©2018 (C) Progger
©2018 Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/L68Y/fJZkA3mYf

Created on October 5, 2018

I checked this part of the database (9294 CFs ODLS) and found 10 duplicates

0 2 3 4 5 7 8 6 9 1
3 1 4 6 9 0 5 8 2 7
1 5 2 0 6 9 7 3 4 8
2 7 8 3 1 4 0 9 6 5
9 0 7 8 4 6 1 2 5 3
4 8 1 9 7 5 2 0 3 6
7 9 0 5 3 8 6 4 1 2
5 6 9 2 8 1 3 7 0 4
6 4 5 7 2 3 9 1 8 0
8 3 6 1 0 2 4 5 7 9

0 2 3 4 5 7 8 6 9 1
3 1 8 6 9 0 5 4 2 7
1 5 2 0 6 9 7 3 4 8
2 7 4 3 1 8 0 9 6 5
9 0 7 8 4 6 1 2 5 3
4 8 1 9 7 5 2 0 3 6
7 9 0 5 3 4 6 8 1 2
5 6 9 2 8 1 3 7 0 4
6 4 5 7 2 3 9 1 8 0
8 3 6 1 0 2 4 5 7 9

0 7 8 6 2 9 3 5 4 1
6 1 5 0 8 3 9 4 2 7
7 6 2 8 5 4 1 3 9 0
4 9 7 3 1 8 0 6 5 2
5 3 1 9 4 6 7 2 0 8
2 0 4 1 7 5 8 9 6 3
9 2 3 5 0 1 6 8 7 4
1 8 9 2 6 0 4 7 3 5
3 4 0 7 9 2 5 1 8 6
8 5 6 4 3 7 2 0 1 9

0 9 8 7 6 4 5 2 3 1
6 1 3 2 8 7 9 5 0 4
7 4 2 6 0 8 1 3 9 5
8 5 1 3 9 0 4 6 7 2
1 7 5 0 4 6 3 9 2 8
3 6 4 9 7 5 2 8 1 0
9 3 0 8 1 2 6 4 5 7
2 0 9 4 5 1 8 7 6 3
4 2 7 5 3 9 0 1 8 6
5 8 6 1 2 3 7 0 4 9

0 2 3 5 6 4 7 8 9 1
2 1 4 7 5 9 3 6 0 8
6 0 2 8 1 7 9 3 4 5
9 6 8 3 0 1 2 5 7 4
7 8 9 1 4 6 5 2 3 0
4 3 1 9 7 5 8 0 6 2
1 7 0 4 8 2 6 9 5 3
3 4 5 2 9 8 0 7 1 6
5 9 6 0 2 3 1 4 8 7
8 5 7 6 3 0 4 1 2 9

0 2 3 5 6 4 7 8 9 1
2 1 8 7 3 9 4 6 0 5
9 7 2 8 0 1 5 3 4 6
5 4 6 3 8 7 2 9 1 0
3 8 1 9 4 6 0 2 5 7
6 0 9 1 7 5 8 4 2 3
1 5 7 4 9 2 6 0 3 8
4 3 5 0 1 8 9 7 6 2
7 9 0 6 2 3 1 5 8 4
8 6 4 2 5 0 3 1 7 9

0 4 5 7 3 8 9 2 6 1
8 1 7 9 0 3 4 5 2 6
5 6 2 0 1 7 8 3 9 4
1 2 4 3 6 9 7 0 5 8
9 7 3 8 4 6 2 1 0 5
3 9 6 2 8 5 0 4 1 7
7 3 0 5 9 1 6 8 4 2
6 8 9 4 5 2 1 7 3 0
2 0 1 6 7 4 5 9 8 3
4 5 8 1 2 0 3 6 7 9

0 5 4 2 7 3 9 8 6 1
6 1 3 8 0 9 5 4 2 7
8 7 2 4 5 0 1 3 9 6
9 6 0 3 2 7 4 5 1 8
1 3 7 0 4 6 8 9 5 2
2 4 9 1 8 5 7 6 0 3
5 2 8 9 3 1 6 0 7 4
3 9 5 6 1 8 2 7 4 0
4 0 6 7 9 2 3 1 8 5
7 8 1 5 6 4 0 2 3 9

0 2 3 6 5 4 8 9 7 1
4 1 7 5 8 9 3 6 0 2
1 8 2 9 0 7 4 3 6 5
9 7 1 3 6 0 2 5 4 8
8 3 9 2 4 6 1 0 5 7
2 4 0 8 7 5 9 1 3 6
7 0 5 4 2 1 6 8 9 3
6 5 8 1 9 3 0 7 2 4
3 9 6 7 1 2 5 4 8 0
5 6 4 0 3 8 7 2 1 9

0 2 6 9 5 8 7 4 3 1
4 1 9 5 7 6 8 3 2 0
3 6 2 7 8 9 1 0 4 5
5 0 8 3 6 1 4 2 9 7
8 9 3 1 4 7 5 6 0 2
2 4 7 0 9 5 3 1 6 8
1 7 4 8 2 0 6 9 5 3
9 8 5 6 0 3 2 7 1 4
7 3 0 4 1 2 9 5 8 6
6 5 1 2 3 4 0 8 7 9

Tomas Brada
did you find the same duplicates?

CFs LS may be less than CFs DLS. It is right.

First step in zamyk.bat is converting the input into CF LK. Can that result in a loss of solutions?

CFs LS may be less than CFs DLS. It is right.

First step in zamyk.bat is converting the input into CF LK. Can that result in a loss of solutions?

I do not know the work of the program zamyk.bat.
This is a question to Belyshev.

However, the zamyk.bat script seems to work correctly.
I have not noticed any errors in the work of this program.

There are 12 KF ODLK duplicates from which 10 are literal. See this report:

Processing DB_500000.txt : 500000
DB_500000.txt -: in: 500000 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 500000
db unique kf dlk: 500000
Processing DB_part2_500000.txt : 500000
DB_part2_500000.txt -: in: 500000 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 500000
db unique kf dlk: 1000000
Processing DB_part3_500000.txt : 500000
DB_part3_500000.txt -: in: 500000 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 500000
db unique kf dlk: 1500000
Processing DB_part4_500000.txt : 500000
DB_part4_500000.txt -: in: 500000 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 500000
db unique kf dlk: 2000000
Processing addition_results_67879.txt : 67879
addition_results_67879.txt -: in: 67879 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 67879
db unique kf dlk: 2067879
Processing add_rez_9294.txt : 9294
add_rez_9294.txt -: in: 9294 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 9294
db unique kf dlk: 2077173
Processing rez_ODLK1_may_F1.txt : 447612
duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 6 9 4 8 5 0
4 8 6 0 5 3 9 2 1 7
2 5 9 8 7 6 3 4 0 1
7 3 1 6 9 8 5 0 4 2
8 6 4 5 1 7 0 9 2 3
3 7 0 4 8 2 1 6 9 5
9 4 7 1 3 0 2 5 6 8
5 0 8 9 2 4 7 1 3 6
6 9 5 2 0 1 8 3 7 4

kf (lin=65) of the duplicate:
0 7 8 6 2 9 3 5 4 1
6 1 5 0 8 3 9 4 2 7
7 6 2 8 5 4 1 3 9 0
4 9 7 3 1 8 0 6 5 2
5 3 1 9 4 6 7 2 0 8
2 0 4 1 7 5 8 9 6 3
9 2 3 5 0 1 6 8 7 4
1 8 9 2 6 0 4 7 3 5
3 4 0 7 9 2 5 1 8 6
8 5 6 4 3 7 2 0 1 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 8 5 6 7 9 4 0
4 6 9 1 7 2 5 8 0 3
8 3 7 5 0 9 1 6 2 4
9 8 4 7 6 3 2 0 5 1
7 9 5 0 2 8 3 4 1 6
3 0 8 6 9 1 4 2 7 5
6 5 0 4 3 7 8 1 9 2
2 7 6 9 1 4 0 5 3 8
5 4 1 2 8 0 9 3 6 7

kf (lin=65) of the duplicate:
0 2 3 4 5 7 8 6 9 1
3 1 4 6 9 0 5 8 2 7
1 5 2 0 6 9 7 3 4 8
2 7 8 3 1 4 0 9 6 5
9 0 7 8 4 6 1 2 5 3
4 8 1 9 7 5 2 0 3 6
7 9 0 5 3 8 6 4 1 2
5 6 9 2 8 1 3 7 0 4
6 4 5 7 2 3 9 1 8 0
8 3 6 1 0 2 4 5 7 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 8 5 6 9 0 4 7
9 8 7 2 0 3 5 6 1 4
5 0 8 6 1 4 7 3 9 2
7 4 6 1 9 8 3 2 5 0
4 7 5 9 3 1 0 8 2 6
3 9 4 0 6 2 8 5 7 1
8 6 1 5 7 9 2 4 0 3
6 5 0 4 2 7 1 9 3 8
2 3 9 7 8 0 4 1 6 5

kf (lin=58) of the duplicate:
0 9 8 7 6 4 5 2 3 1
6 1 3 2 8 7 9 5 0 4
7 4 2 6 0 8 1 3 9 5
8 5 1 3 9 0 4 6 7 2
1 7 5 0 4 6 3 9 2 8
3 6 4 9 7 5 2 8 1 0
9 3 0 8 1 2 6 4 5 7
2 0 9 4 5 1 8 7 6 3
4 2 7 5 3 9 0 1 8 6
5 8 6 1 2 3 7 0 4 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 9 6 8 4 0 5 7
4 7 6 2 5 1 9 3 0 8
8 0 4 5 2 9 7 1 6 3
9 3 8 1 7 6 5 2 4 0
7 6 5 8 0 3 1 4 9 2
5 9 0 4 3 2 8 6 7 1
6 4 1 7 8 0 3 9 2 5
3 8 7 0 9 4 2 5 1 6
2 5 9 6 1 7 0 8 3 4

kf (lin=65) of the duplicate:
0 2 3 4 5 7 8 6 9 1
3 1 8 6 9 0 5 4 2 7
1 5 2 0 6 9 7 3 4 8
2 7 4 3 1 8 0 9 6 5
9 0 7 8 4 6 1 2 5 3
4 8 1 9 7 5 2 0 3 6
7 9 0 5 3 4 6 8 1 2
5 6 9 2 8 1 3 7 0 4
6 4 5 7 2 3 9 1 8 0
8 3 6 1 0 2 4 5 7 9

rez_ODLK1_may_F1.txt -: in: 447612 not lk: 0 not dlk: 0 dup: 4 new uniqe kf dlk: 447608
db unique kf dlk: 2524781
Processing rez_ODLK1_Jun_F1.txt : 486061
duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 5 8 9 4 6 0
8 6 9 4 1 0 5 3 7 2
5 9 0 1 6 4 7 2 3 8
6 0 5 8 7 2 3 1 9 4
3 7 4 9 0 6 2 8 5 1
9 4 7 5 2 3 8 0 1 6
2 3 1 6 8 9 4 5 0 7
7 8 6 2 3 1 0 9 4 5
4 5 8 0 9 7 1 6 2 3

kf (lin=54) of the duplicate:
0 4 5 7 3 8 9 2 6 1
8 1 7 9 0 3 4 5 2 6
5 6 2 0 1 7 8 3 9 4
1 2 4 3 6 9 7 0 5 8
9 7 3 8 4 6 2 1 0 5
3 9 6 2 8 5 0 4 1 7
7 3 0 5 9 1 6 8 4 2
6 8 9 4 5 2 1 7 3 0
2 0 1 6 7 4 5 9 8 3
4 5 8 1 2 0 3 6 7 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 5 8 9 6 4 0
8 7 4 2 9 3 0 5 1 6
4 8 5 9 0 6 2 3 7 1
5 3 8 4 6 1 7 0 9 2
9 6 0 1 3 7 5 4 2 8
7 5 6 8 2 0 1 9 3 4
2 9 7 0 1 4 3 8 6 5
3 0 1 6 8 9 4 2 5 7
6 4 9 5 7 2 8 1 0 3

kf (lin=53) of the duplicate:
0 5 4 2 7 3 9 8 6 1
6 1 3 8 0 9 5 4 2 7
8 7 2 4 5 0 1 3 9 6
9 6 0 3 2 7 4 5 1 8
1 3 7 0 4 6 8 9 5 2
2 4 9 1 8 5 7 6 0 3
5 2 8 9 3 1 6 0 7 4
3 9 5 6 1 8 2 7 4 0
4 0 6 7 9 2 3 1 8 5
7 8 1 5 6 4 0 2 3 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 6 9 8 5 4 0
5 7 4 0 8 6 9 2 1 3
2 5 9 6 7 1 3 4 0 8
4 8 7 2 9 0 5 3 6 1
8 3 0 4 5 7 2 1 9 6
3 9 5 8 0 4 1 6 7 2
6 0 1 9 2 3 7 8 5 4
9 6 8 5 1 2 4 0 3 7
7 4 6 1 3 8 0 9 2 5

kf (lin=0) of the duplicate:
0 2 3 5 6 4 7 8 9 1
2 1 8 7 3 9 4 6 0 5
9 7 2 8 0 1 5 3 4 6
5 4 6 3 8 7 2 9 1 0
3 8 1 9 4 6 0 2 5 7
6 0 9 1 7 5 8 4 2 3
1 5 7 4 9 2 6 0 3 8
4 3 5 0 1 8 9 7 6 2
7 9 0 6 2 3 1 5 8 4
8 6 4 2 5 0 3 1 7 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 8 0 9 5 4 7 6
3 9 5 7 2 4 8 6 0 1
2 5 0 4 1 8 3 9 6 7
5 4 6 9 7 2 1 0 3 8
8 7 4 1 5 6 0 3 9 2
6 3 7 0 8 1 9 5 2 4
9 6 1 2 3 0 7 8 4 5
7 8 9 5 6 3 4 2 1 0
4 0 8 6 9 7 2 1 5 3

kf (lin=0) of the duplicate:
0 2 3 5 6 4 7 8 9 1
2 1 4 7 5 9 3 6 0 8
6 0 2 8 1 7 9 3 4 5
9 6 8 3 0 1 2 5 7 4
7 8 9 1 4 6 5 2 3 0
4 3 1 9 7 5 8 0 6 2
1 7 0 4 8 2 6 9 5 3
3 4 5 2 9 8 0 7 1 6
5 9 6 0 2 3 1 4 8 7
8 5 7 6 3 0 4 1 2 9

rez_ODLK1_Jun_F1.txt -: in: 486061 not lk: 0 not dlk: 0 dup: 4 new uniqe kf dlk: 486057
db unique kf dlk: 3010838
Processing rez_ODLK1_Jul_F1.txt : 407150
duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 6 9 0 8 4 5
8 6 1 9 5 4 7 0 3 2
6 0 9 8 3 2 1 4 5 7
5 7 6 4 9 3 8 2 1 0
3 4 7 5 2 6 9 1 0 8
2 8 5 6 7 0 4 3 9 1
9 3 8 1 0 7 2 5 6 4
4 5 0 2 8 1 3 9 7 6
7 9 4 0 1 8 5 6 2 3

kf (lin=44) of the duplicate:
0 2 6 9 5 8 7 4 3 1
4 1 9 5 7 6 8 3 2 0
3 6 2 7 8 9 1 0 4 5
5 0 8 3 6 1 4 2 9 7
8 9 3 1 4 7 5 6 0 2
2 4 7 0 9 5 3 1 6 8
1 7 4 8 2 0 6 9 5 3
9 8 5 6 0 3 2 7 1 4
7 3 0 4 1 2 9 5 8 6
6 5 1 2 3 4 0 8 7 9

rez_ODLK1_Jul_F1.txt -: in: 407150 not lk: 0 not dlk: 0 dup: 1 new uniqe kf dlk: 407149
db unique kf dlk: 3417987
Processing rez_ODLK1_aug.txt : 400553
duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 8 6 9 4 0 5 7
5 9 4 7 0 8 2 3 6 1
4 5 9 6 2 3 7 8 1 0
8 3 0 9 7 6 5 1 2 4
7 0 6 4 8 1 3 2 9 5
3 8 5 0 1 2 9 4 7 6
6 7 1 2 9 0 8 5 4 3
9 4 8 1 5 7 0 6 3 2
2 6 7 5 3 4 1 9 0 8

kf (lin=1) of the duplicate:
0 2 3 6 5 4 8 9 7 1
4 1 7 5 8 9 3 6 0 2
1 8 2 9 0 7 4 3 6 5
9 7 1 3 6 0 2 5 4 8
8 3 9 2 4 6 1 0 5 7
2 4 0 8 7 5 9 1 3 6
7 0 5 4 2 1 6 8 9 3
6 5 8 1 9 3 0 7 2 4
3 9 6 7 1 2 5 4 8 0
5 6 4 0 3 8 7 2 1 9

rez_ODLK1_aug.txt -: in: 400553 not lk: 0 not dlk: 0 dup: 1 new uniqe kf dlk: 400552
db unique kf dlk: 3818539
Processing rez_ODLK1_Sep_F2.txt : 407995
rez_ODLK1_Sep_F2.txt -: in: 407995 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 407995
db unique kf dlk: 4226534
Processing rez_ODLK1_Oct_F2.txt : 404478
duplicate (orig) :
0 2 3 4 5 7 8 6 9 1
6 1 5 8 3 9 7 4 0 2
5 4 2 6 0 1 9 3 7 8
7 6 4 3 9 8 2 1 5 0
3 8 0 7 4 6 1 9 2 5
9 3 1 2 7 5 0 8 4 6
8 7 9 5 1 2 6 0 3 4
2 0 8 9 6 4 5 7 1 3
1 9 6 0 2 3 4 5 8 7
4 5 7 1 8 0 3 2 6 9

duplicate (orig) :
0 3 4 8 5 2 9 6 7 1
3 1 5 4 6 9 7 2 0 8
5 0 2 6 9 8 1 3 4 7
7 8 1 3 0 4 2 9 6 5
2 5 3 7 4 6 8 1 9 0
1 6 0 9 7 5 4 8 3 2
8 7 9 5 3 1 6 0 2 4
9 4 8 1 2 3 0 7 5 6
6 9 7 2 1 0 5 4 8 3
4 2 6 0 8 7 3 5 1 9

rez_ODLK1_Oct_F2.txt -: in: 404478 not lk: 0 not dlk: 0 dup: 2 new uniqe kf dlk: 404476
db unique kf dlk: 4631010
Processing rez_ODLK1_Nov_F2.txt : 355634
rez_ODLK1_Nov_F2.txt -: in: 355634 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 355634
db unique kf dlk: 4986644
Processing rez_ODLK1_Dec_F2.txt : 343240
rez_ODLK1_Dec_F2.txt -: in: 343240 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 343240
db unique kf dlk: 5329884

There are
4 in rez_ODLK1_may_F1.txt,
4 in rez_ODLK1_Jun_F1.txt,
1 in rez_ODLK1_Jul_F1.txt,
1 in rez_ODLK1_aug.txt, and
2 in rez_ODLK1_Oct_F2.txt.

---

Tomas Brada
please show two DLS except 10 DLS, which I showed above.

I removed 10 DLS from part 9 (9294 CFs ODLS), now this part contains 9284 CFs.

Further, I do not see which parts still contain duplicates.

In the log published by you the checking goes in a different order.

I reordered the files and this is the new report.

Processing DB_500000.txt : 500000
DB_500000.txt -: in: 500000 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 500000
db unique kf dlk: 500000
Processing DB_part2_500000.txt : 500000
DB_part2_500000.txt -: in: 500000 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 500000
db unique kf dlk: 1000000
Processing DB_part3_500000.txt : 500000
DB_part3_500000.txt -: in: 500000 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 500000
db unique kf dlk: 1500000
Processing DB_part4_500000.txt : 500000
DB_part4_500000.txt -: in: 500000 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 500000
db unique kf dlk: 2000000
Processing rez_ODLK1_may_F1.txt : 447612
rez_ODLK1_may_F1.txt -: in: 447612 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 447612
db unique kf dlk: 2447612
Processing rez_ODLK1_Jun_F1.txt : 486061
rez_ODLK1_Jun_F1.txt -: in: 486061 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 486061
db unique kf dlk: 2933673
Processing rez_ODLK1_Jul_F1.txt : 407150
rez_ODLK1_Jul_F1.txt -: in: 407150 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 407150
db unique kf dlk: 3340823
Processing rez_ODLK1_aug.txt : 400553
rez_ODLK1_aug.txt -: in: 400553 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 400553
db unique kf dlk: 3741376
Processing add_rez_9294.txt : 9294
duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 5 8 9 4 6 0
8 6 9 4 1 0 5 3 7 2
5 9 0 1 6 4 7 2 3 8
6 0 5 8 7 2 3 1 9 4
3 7 4 9 0 6 2 8 5 1
9 4 7 5 2 3 8 0 1 6
2 3 1 6 8 9 4 5 0 7
7 8 6 2 3 1 0 9 4 5
4 5 8 0 9 7 1 6 2 3

kf (lin=54) of the duplicate:
0 4 5 7 3 8 9 2 6 1
8 1 7 9 0 3 4 5 2 6
5 6 2 0 1 7 8 3 9 4
1 2 4 3 6 9 7 0 5 8
9 7 3 8 4 6 2 1 0 5
3 9 6 2 8 5 0 4 1 7
7 3 0 5 9 1 6 8 4 2
6 8 9 4 5 2 1 7 3 0
2 0 1 6 7 4 5 9 8 3
4 5 8 1 2 0 3 6 7 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 5 8 9 6 4 0
8 7 4 2 9 3 0 5 1 6
4 8 5 9 0 6 2 3 7 1
5 3 8 4 6 1 7 0 9 2
9 6 0 1 3 7 5 4 2 8
7 5 6 8 2 0 1 9 3 4
2 9 7 0 1 4 3 8 6 5
3 0 1 6 8 9 4 2 5 7
6 4 9 5 7 2 8 1 0 3

kf (lin=53) of the duplicate:
0 5 4 2 7 3 9 8 6 1
6 1 3 8 0 9 5 4 2 7
8 7 2 4 5 0 1 3 9 6
9 6 0 3 2 7 4 5 1 8
1 3 7 0 4 6 8 9 5 2
2 4 9 1 8 5 7 6 0 3
5 2 8 9 3 1 6 0 7 4
3 9 5 6 1 8 2 7 4 0
4 0 6 7 9 2 3 1 8 5
7 8 1 5 6 4 0 2 3 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 6 9 0 8 4 5
8 6 1 9 5 4 7 0 3 2
6 0 9 8 3 2 1 4 5 7
5 7 6 4 9 3 8 2 1 0
3 4 7 5 2 6 9 1 0 8
2 8 5 6 7 0 4 3 9 1
9 3 8 1 0 7 2 5 6 4
4 5 0 2 8 1 3 9 7 6
7 9 4 0 1 8 5 6 2 3

kf (lin=44) of the duplicate:
0 2 6 9 5 8 7 4 3 1
4 1 9 5 7 6 8 3 2 0
3 6 2 7 8 9 1 0 4 5
5 0 8 3 6 1 4 2 9 7
8 9 3 1 4 7 5 6 0 2
2 4 7 0 9 5 3 1 6 8
1 7 4 8 2 0 6 9 5 3
9 8 5 6 0 3 2 7 1 4
7 3 0 4 1 2 9 5 8 6
6 5 1 2 3 4 0 8 7 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 6 9 4 8 5 0
4 8 6 0 5 3 9 2 1 7
2 5 9 8 7 6 3 4 0 1
7 3 1 6 9 8 5 0 4 2
8 6 4 5 1 7 0 9 2 3
3 7 0 4 8 2 1 6 9 5
9 4 7 1 3 0 2 5 6 8
5 0 8 9 2 4 7 1 3 6
6 9 5 2 0 1 8 3 7 4

kf (lin=65) of the duplicate:
0 7 8 6 2 9 3 5 4 1
6 1 5 0 8 3 9 4 2 7
7 6 2 8 5 4 1 3 9 0
4 9 7 3 1 8 0 6 5 2
5 3 1 9 4 6 7 2 0 8
2 0 4 1 7 5 8 9 6 3
9 2 3 5 0 1 6 8 7 4
1 8 9 2 6 0 4 7 3 5
3 4 0 7 9 2 5 1 8 6
8 5 6 4 3 7 2 0 1 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 7 6 9 8 5 4 0
5 7 4 0 8 6 9 2 1 3
2 5 9 6 7 1 3 4 0 8
4 8 7 2 9 0 5 3 6 1
8 3 0 4 5 7 2 1 9 6
3 9 5 8 0 4 1 6 7 2
6 0 1 9 2 3 7 8 5 4
9 6 8 5 1 2 4 0 3 7
7 4 6 1 3 8 0 9 2 5

kf (lin=0) of the duplicate:
0 2 3 5 6 4 7 8 9 1
2 1 8 7 3 9 4 6 0 5
9 7 2 8 0 1 5 3 4 6
5 4 6 3 8 7 2 9 1 0
3 8 1 9 4 6 0 2 5 7
6 0 9 1 7 5 8 4 2 3
1 5 7 4 9 2 6 0 3 8
4 3 5 0 1 8 9 7 6 2
7 9 0 6 2 3 1 5 8 4
8 6 4 2 5 0 3 1 7 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 8 0 9 5 4 7 6
3 9 5 7 2 4 8 6 0 1
2 5 0 4 1 8 3 9 6 7
5 4 6 9 7 2 1 0 3 8
8 7 4 1 5 6 0 3 9 2
6 3 7 0 8 1 9 5 2 4
9 6 1 2 3 0 7 8 4 5
7 8 9 5 6 3 4 2 1 0
4 0 8 6 9 7 2 1 5 3

kf (lin=0) of the duplicate:
0 2 3 5 6 4 7 8 9 1
2 1 4 7 5 9 3 6 0 8
6 0 2 8 1 7 9 3 4 5
9 6 8 3 0 1 2 5 7 4
7 8 9 1 4 6 5 2 3 0
4 3 1 9 7 5 8 0 6 2
1 7 0 4 8 2 6 9 5 3
3 4 5 2 9 8 0 7 1 6
5 9 6 0 2 3 1 4 8 7
8 5 7 6 3 0 4 1 2 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 8 5 6 7 9 4 0
4 6 9 1 7 2 5 8 0 3
8 3 7 5 0 9 1 6 2 4
9 8 4 7 6 3 2 0 5 1
7 9 5 0 2 8 3 4 1 6
3 0 8 6 9 1 4 2 7 5
6 5 0 4 3 7 8 1 9 2
2 7 6 9 1 4 0 5 3 8
5 4 1 2 8 0 9 3 6 7

kf (lin=65) of the duplicate:
0 2 3 4 5 7 8 6 9 1
3 1 4 6 9 0 5 8 2 7
1 5 2 0 6 9 7 3 4 8
2 7 8 3 1 4 0 9 6 5
9 0 7 8 4 6 1 2 5 3
4 8 1 9 7 5 2 0 3 6
7 9 0 5 3 8 6 4 1 2
5 6 9 2 8 1 3 7 0 4
6 4 5 7 2 3 9 1 8 0
8 3 6 1 0 2 4 5 7 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 8 5 6 9 0 4 7
9 8 7 2 0 3 5 6 1 4
5 0 8 6 1 4 7 3 9 2
7 4 6 1 9 8 3 2 5 0
4 7 5 9 3 1 0 8 2 6
3 9 4 0 6 2 8 5 7 1
8 6 1 5 7 9 2 4 0 3
6 5 0 4 2 7 1 9 3 8
2 3 9 7 8 0 4 1 6 5

kf (lin=58) of the duplicate:
0 9 8 7 6 4 5 2 3 1
6 1 3 2 8 7 9 5 0 4
7 4 2 6 0 8 1 3 9 5
8 5 1 3 9 0 4 6 7 2
1 7 5 0 4 6 3 9 2 8
3 6 4 9 7 5 2 8 1 0
9 3 0 8 1 2 6 4 5 7
2 0 9 4 5 1 8 7 6 3
4 2 7 5 3 9 0 1 8 6
5 8 6 1 2 3 7 0 4 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 8 6 9 4 0 5 7
5 9 4 7 0 8 2 3 6 1
4 5 9 6 2 3 7 8 1 0
8 3 0 9 7 6 5 1 2 4
7 0 6 4 8 1 3 2 9 5
3 8 5 0 1 2 9 4 7 6
6 7 1 2 9 0 8 5 4 3
9 4 8 1 5 7 0 6 3 2
2 6 7 5 3 4 1 9 0 8

kf (lin=1) of the duplicate:
0 2 3 6 5 4 8 9 7 1
4 1 7 5 8 9 3 6 0 2
1 8 2 9 0 7 4 3 6 5
9 7 1 3 6 0 2 5 4 8
8 3 9 2 4 6 1 0 5 7
2 4 0 8 7 5 9 1 3 6
7 0 5 4 2 1 6 8 9 3
6 5 8 1 9 3 0 7 2 4
3 9 6 7 1 2 5 4 8 0
5 6 4 0 3 8 7 2 1 9

duplicate (orig) :
0 1 2 3 4 5 6 7 8 9
1 2 3 9 6 8 4 0 5 7
4 7 6 2 5 1 9 3 0 8
8 0 4 5 2 9 7 1 6 3
9 3 8 1 7 6 5 2 4 0
7 6 5 8 0 3 1 4 9 2
5 9 0 4 3 2 8 6 7 1
6 4 1 7 8 0 3 9 2 5
3 8 7 0 9 4 2 5 1 6
2 5 9 6 1 7 0 8 3 4

kf (lin=65) of the duplicate:
0 2 3 4 5 7 8 6 9 1
3 1 8 6 9 0 5 4 2 7
1 5 2 0 6 9 7 3 4 8
2 7 4 3 1 8 0 9 6 5
9 0 7 8 4 6 1 2 5 3
4 8 1 9 7 5 2 0 3 6
7 9 0 5 3 4 6 8 1 2
5 6 9 2 8 1 3 7 0 4
6 4 5 7 2 3 9 1 8 0
8 3 6 1 0 2 4 5 7 9

add_rez_9294.txt -: in: 9294 not lk: 0 not dlk: 0 dup: 10 new uniqe kf dlk: 9284
db unique kf dlk: 3750660
Processing addition_results_67879.txt : 67879
addition_results_67879.txt -: in: 67879 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 67879
db unique kf dlk: 3818539
Processing rez_ODLK1_Sep_F2.txt : 407995
rez_ODLK1_Sep_F2.txt -: in: 407995 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 407995
db unique kf dlk: 4226534
Processing rez_ODLK1_Oct_F2.txt : 404478
duplicate (orig) :
0 2 3 4 5 7 8 6 9 1
6 1 5 8 3 9 7 4 0 2
5 4 2 6 0 1 9 3 7 8
7 6 4 3 9 8 2 1 5 0
3 8 0 7 4 6 1 9 2 5
9 3 1 2 7 5 0 8 4 6
8 7 9 5 1 2 6 0 3 4
2 0 8 9 6 4 5 7 1 3
1 9 6 0 2 3 4 5 8 7
4 5 7 1 8 0 3 2 6 9

duplicate (orig) :
0 3 4 8 5 2 9 6 7 1
3 1 5 4 6 9 7 2 0 8
5 0 2 6 9 8 1 3 4 7
7 8 1 3 0 4 2 9 6 5
2 5 3 7 4 6 8 1 9 0
1 6 0 9 7 5 4 8 3 2
8 7 9 5 3 1 6 0 2 4
9 4 8 1 2 3 0 7 5 6
6 9 7 2 1 0 5 4 8 3
4 2 6 0 8 7 3 5 1 9

rez_ODLK1_Oct_F2.txt -: in: 404478 not lk: 0 not dlk: 0 dup: 2 new uniqe kf dlk: 404476
db unique kf dlk: 4631010
Processing rez_ODLK1_Nov_F2.txt : 355634
rez_ODLK1_Nov_F2.txt -: in: 355634 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 355634
db unique kf dlk: 4986644
Processing rez_ODLK1_Dec_F2.txt : 343240
rez_ODLK1_Dec_F2.txt -: in: 343240 not lk: 0 not dlk: 0 dup: 0 new uniqe kf dlk: 343240
db unique kf dlk: 5329884