The report shows first which file it is processing and how many squares were in the file, then if duplicate is found, it will show the square as in the file and if that square is not diagonal cf, then also kf of the duplicate.

Are these duplicates in the rez_ODLK1_Oct_F2.txt part???

Processing rez_ODLK1_Oct_F2.txt : 404478
duplicate (orig) :
0 2 3 4 5 7 8 6 9 1
6 1 5 8 3 9 7 4 0 2
5 4 2 6 0 1 9 3 7 8
7 6 4 3 9 8 2 1 5 0
3 8 0 7 4 6 1 9 2 5
9 3 1 2 7 5 0 8 4 6
8 7 9 5 1 2 6 0 3 4
2 0 8 9 6 4 5 7 1 3
1 9 6 0 2 3 4 5 8 7
4 5 7 1 8 0 3 2 6 9

duplicate (orig) :
0 3 4 8 5 2 9 6 7 1
3 1 5 4 6 9 7 2 0 8
5 0 2 6 9 8 1 3 4 7
7 8 1 3 0 4 2 9 6 5
2 5 3 7 4 6 8 1 9 0
1 6 0 9 7 5 4 8 3 2
8 7 9 5 3 1 6 0 2 4
9 4 8 1 2 3 0 7 5 6
6 9 7 2 1 0 5 4 8 3
4 2 6 0 8 7 3 5 1 9

rez_ODLK1_Oct_F2.txt -: in: 404478 not lk: 0 not dlk: 0 dup: 2 new uniqe kf dlk: 404476

But in the rez_ODLK1_Oct_F2.txt part there are no two such DLS!

In addition, the kanonizator_dlk_1.03 program reports that in this part there are 404478 CFs.

Канонизатор ДЛК10

Загружена хеш-таблица
Время загрузки:         0.015 сек
Введено ДЛК:            404478
Время загрузки:         4.758 сек
Найдено КФ:             404478
Время поиска:           0.546 сек
КФ записаны в файл:     output.txt
Время записи:           0.375 сек
Общее время работы:     5.694 сек

rez_ODLK1_Oct_F2.txt -: in: 404478 not lk: 0 not dlk: 0 dup: 2 new uniqe kf dlk: 404476

Maybe these “2 new uniqe kf dlk” are contained in the previous parts?

In which parts?

Tomas Brada
I understand that you should not solve this problem.
Excuse me.

I wanted to understand where the error is.
But I do not understand this :(

The problem is that I cannot merge all the parts into one file.
Therefore, I check each part separately.

The program works across all files. It goes like this: first file is read, the cf dls are added to memory set. Then the second file is read, each square is converted to cf dlk and checked against the memory set if it already exists there, if not that means it is unique and is added to memory set, if it already is in the memory set, this duplicate is reported and not added again.

rez_ODLK1_Oct_F2.txt -: in: 404478 not lk: 0 not dlk: 0 dup: 2 new uniqe kf dlk: 404476

rez_ODLK1_Oct_F2.txt - input file
in: 404478 - 404478 squares in input file
not lk: 0 - no squares failed the is_lk() check
not dlk: 0 - no squares failed is_dlk() check
dup: 2 - two duplicates
new uniqe kf dlk: 404476 - 404476 cf dls added to memory set

That means there are two KF DLK which already are in at least one of the previous files.

That means there are two KF DLK which already are in at least one of the previous files.

The fact of the matter is that I did not find these two DLS in previous files!

Can you tell exactly in which part are these two DLS?

I see these two DLS only in the rez_ODLK1_Oct_F2.txt part.

I found!

These two DLS are contained in the part addition_results_67879.txt.
No more questions.

Thanks.

I published the results of the project ODLK for May - December 2018
https://cloud.mail.ru/public/6igF/E4JFS4ZYy

385041 unique CFs ODLS

I checked the results for uniqueness regarding the results of the project ODLK1.

Tomas Brada
can you check these results for uniqueness with respect to the database of 5,329,884 GFs?

db unique kf dlk: 5329884
Processing rez_ODLK_May-dec_F2.txt : 385041
rez_ODLK_May-dec_F2.txt -: in: 385041, not lk: 0, not dlk: 0, dup: 0, new uniqe kf dlk: 385041                           
db unique kf dlk: 5714925

No duplicates with respect to the 5329884 database. 385041 new. Now 5714925 in the database.

db unique kf dlk: 5329884
Processing rez_ODLK_May-dec_F2.txt : 385041
rez_ODLK_May-dec_F2.txt -: in: 385041, not lk: 0, not dlk: 0, dup: 0, new uniqe kf dlk: 385041                           
db unique kf dlk: 5714925

No duplicates with respect to the 5329884 database. 385041 new. Now 5714925 in the database.

Thank you very much!

Fine!

I introducing the new DLS recording format by Belyshev
http://forum.boinc.ru/default.aspx?g=posts&m=90094#post90094

Download the namer.zip archive
Further you watch the redme.txt file

Example

0 2 4 5 7 3 9 8 6 1
8 1 6 4 5 2 3 9 0 7
7 8 2 6 0 4 1 3 9 5
1 9 7 3 2 8 4 6 5 0
9 5 3 0 4 6 2 1 7 8
4 6 1 9 8 5 7 0 3 2
3 0 8 2 9 7 6 5 1 4
6 4 9 8 1 0 5 7 2 3
2 7 5 1 3 9 0 4 8 6
5 3 0 7 6 1 8 2 4 9

0 3 7 6 5 9 2 8 4 1
3 1 6 2 9 8 7 5 0 4
9 6 2 1 0 4 8 3 5 7
6 2 1 3 8 0 4 9 7 5
5 8 0 7 4 2 9 1 6 3
2 7 3 0 6 5 1 4 9 8
4 9 5 8 3 7 6 0 1 2
8 4 9 5 1 3 0 7 2 6
1 5 4 9 7 6 3 2 8 0
7 0 8 4 2 1 5 6 3 9

0 4 5 7 8 6 9 2 3 1
5 1 3 8 7 4 2 9 0 6
8 7 2 9 0 1 5 3 6 4
1 9 8 3 5 2 4 6 7 0
3 5 0 6 4 8 7 1 9 2
6 8 7 2 9 5 0 4 1 3
2 3 9 5 1 7 6 0 4 8
4 0 6 1 3 9 8 7 2 5
9 2 1 4 6 0 3 5 8 7
7 6 4 0 2 3 1 8 5 9

0 4 5 8 6 9 7 2 3 1
8 1 6 4 9 2 3 5 0 7
9 6 2 7 0 1 5 3 4 8
2 5 7 3 1 8 4 0 9 6
7 3 0 9 4 6 8 1 2 5
6 9 1 0 8 5 2 4 7 3
3 0 8 2 5 7 6 9 1 4
4 8 9 6 3 0 1 7 5 2
1 7 4 5 2 3 9 6 8 0
5 2 3 1 7 4 0 8 6 9

0 4 8 7 5 6 3 2 9 1
8 1 7 4 9 3 5 6 0 2
9 8 2 5 7 0 1 3 6 4
5 6 0 3 8 1 4 9 2 7
6 9 5 1 4 2 7 0 3 8
4 2 1 9 6 5 0 8 7 3
2 7 4 8 3 9 6 5 1 0
3 0 9 6 1 8 2 7 4 5
1 5 3 2 0 7 9 4 8 6
7 3 6 0 2 4 8 1 5 9

0 4 9 8 5 7 2 6 3 1
9 1 8 4 6 2 3 5 0 7
8 5 2 6 0 1 9 3 7 4
5 7 1 3 8 0 4 2 9 6
3 9 0 7 4 8 5 1 6 2
6 8 3 2 9 5 7 4 1 0
2 3 7 5 1 9 6 0 4 8
1 0 6 9 3 4 8 7 2 5
4 2 5 0 7 6 1 9 8 3
7 6 4 1 2 3 0 8 5 9

0 5 4 8 2 3 7 6 9 1
3 1 7 6 9 8 2 5 0 4
4 8 2 7 0 1 9 3 5 6
1 6 8 3 5 7 4 9 2 0
2 9 0 5 4 6 8 1 7 3
7 0 3 9 8 5 1 4 6 2
8 3 5 2 1 9 6 0 4 7
6 2 9 0 3 4 5 7 1 8
9 7 1 4 6 0 3 2 8 5
5 4 6 1 7 2 0 8 3 9

0 5 4 8 6 7 3 9 2 1
8 1 9 4 7 3 2 5 0 6
9 0 2 1 8 4 7 3 6 5
6 9 8 3 0 2 4 1 5 7
5 6 3 9 4 8 1 2 7 0
1 4 7 6 9 5 0 8 3 2
7 2 0 5 3 1 6 4 9 8
4 8 6 0 2 9 5 7 1 3
3 7 1 2 5 6 9 0 8 4
2 3 5 7 1 0 8 6 4 9

0 6 7 4 8 9 3 2 5 1
9 1 4 7 0 3 5 6 2 8
4 5 2 0 9 8 1 3 6 7
2 8 0 3 5 7 4 1 9 6
7 2 3 8 4 6 0 9 1 5
6 3 1 2 7 5 9 8 4 0
1 9 8 5 3 2 6 0 7 4
5 4 9 6 2 1 8 7 0 3
3 0 6 9 1 4 7 5 8 2
8 7 5 1 6 0 2 4 3 9

0 8 4 7 2 3 5 6 9 1
9 1 3 6 7 4 8 5 2 0
4 5 2 1 0 9 7 3 6 8
1 9 7 3 6 8 4 2 0 5
2 7 0 8 4 6 9 1 5 3
3 4 6 2 8 5 0 9 1 7
5 2 1 9 3 0 6 8 7 4
8 3 5 0 9 1 2 7 4 6
6 0 9 5 1 7 3 4 8 2
7 6 8 4 5 2 1 0 3 9

These DLSs in new format

7TQf7wSBLMRgpXjfS72KiPwQ3
7c1cX663mMc6FYAPvGsBDVBHB
C2q8vwLHiRe3Cr2aZM2ziWbmh
C8VTvQa4q74CgWQTHDdZgJDmj
CDwdqJLJyM17qm6UoGEbQBibi
GxUE4weihQzxenLNTJDaEHVUC
HDhB2Vk3s5zzvXoJg9sq1f3SQ
HniXR5hQT5sRfULymSTk4gCmz
TuFMHpbuxQBVbcS8g5yt74FHw
UaaA2GbMf74zeVM67NkYou6yi

Important note!
The program namer.exe works only with DLS in format 2.

Do you think it is worth the hassle to introduce new format?

Btw: good compression ratio archived! To encode arbitrary (non-latin) 10x10 square of 0-9 it would take 56 characters. This is only 25.

Do you think it is worth the hassle to introduce new format?

Yes, in the future, necessarily, because the database is increasing.

Can you transfer to me or ice00 DB from 5714925 CFs in a new format?
How it is MB?

ice00
January ends.
Are you ready to process results with software developed by Tomas Brada?
Can you get the entire database from 5714925 unique CFs ODLS to check for new results for uniqueness?
Or did you not do anything to solve this problem?

Will you still publish raw results?

I'm actually too busy with working issues to being able to code/test something in free time due to stringent deadlines I have.

Providing raw not processed result is temporary a workaround that with the script already done it takes only 1 minute.

I'm actually too busy with working issues to being able to code/test something in free time due to stringent deadlines I have.

In this case, you do not need to administer the BOINC project!
I suggest you stop it.

Providing raw not processed result is temporary a workaround that with the script already done it takes only 1 minute.

Temporary a workaround? How long? For 2 years? For 5 years?
You already for 4 months published the raw results.
January is the fifth month.
Who will process the results?
I can't do it on my PC.

There are two administrators in the ODLK1 project.
The results are not processed! New Applications do not run!

Where is the work of administrators???

I had the complete sources and I'm still creating and testing the scripts needed to automate the process (that involve two servers).
Do not expect (automatic processed) result published until Christmas due to my working deadlines and two intensive courses I'm making every days (70+ hours of courses to make in two months, to be followed not during working time).

I can instead publish unprocessed data tomorrow.

ice00
by next Christmas, will you automatically processing the results? :)

It is two weeks that I'm working on it. Being testing some new stuff to be sure it process all in the right way.

It is two weeks that I'm working on it.

And what are your successes?
Can you process the results for January - March 2019 with the script of Belyshev zamyk.bat?
So far you have posted only the raw results for this period.

First step achieved: having only one file from ODLK1 with unique squares by automate the process of adding new month result of the server.

For the periods of 2018-05 .. 2019-03 ODLK1 has: 2086518 unique CFs ODLS.

Step 2 with Belyshev zamyk.bat equivalent processing will follow as soon as it works correctly.