Thread 'Autonomous Subproject'

Message boards : News : Autonomous Subproject
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 15 · Next

AuthorMessage
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 449 - Posted: 19 Dec 2018, 3:18:26 UTC

Tomas Brada, ice00
I invite you to discuss the launch of this experiment in the ODLK1 BOINC project.
I think you can do it together.

ice00
you are already ready for this experiment, only on the Linux system.
Well, let it be Application for the Linux system.
ID: 449 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 450 - Posted: 19 Dec 2018, 3:26:48 UTC

I'm trying to find the best way to organize a subproject.

I already wrote that it makes no sense to check each block of tasks twice.
Many blocks are checked twice. It is not rational.
ID: 450 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tomas Brada

Send message
Joined: 22 Jan 18
Posts: 63
Credit: 373,915
RAC: 0
Message 451 - Posted: 19 Dec 2018, 9:06:16 UTC - in response to Message 443.  

Now you need to check the names_44323p_21332.txt file.
All families are different and do not give the same tasks.

Thank you. I queued up IAEDFXNKG..IAFECTKVH, estimated time 10 hours.
ID: 451 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tomas Brada

Send message
Joined: 22 Jan 18
Posts: 63
Credit: 373,915
RAC: 0
Message 452 - Posted: 19 Dec 2018, 9:59:59 UTC

I share the script that I use.
#!/bin/bash
# Open a log file log.txt
exec 4>>log.txt
echo "$PWD $0 $@" >&4
uname -a >&4
wine --version >&4
date >&4

function gencheck()
{
  # Function to generate and check families in parallel
  # $1 -> file name of family names
  echo "Starting batch $1"
  # The families to check are read from file names_$1.txt
  families=( $(cat names_$1.txt))
  echo " count= ${#families[*]}"
  running=0
  for fam in "${families[@]}"; do
    # Check each family in clean directory
    dir=wu_$1/$fam
    echo "Generate family $fam"
    rm -rf $dir
    mkdir -p $dir
    cd $dir
    echo "Generator=generator_lk_4_31_31 family=$fam" >>log.txt
    # Link (copy) needed files from top-level to current directory
    ln -t. ../../hash_tabl.bin ../../generator_lk_4_31_31.exe ../../build/family_mar.exe

    # Execute generator
    # Generator runs fast, so I have not recompiled it. I use wine to run it
    if wine ./generator_lk_4_31_31.exe $fam 2>&1 | iconv -f cp1251 >>log.txt; then
      # Rename output of generator to input for checker
      if mv family_4_31_31_${fam}_*.txt input.txt; then
        md5sum input.txt >>log.txt
        # Start a checker program on background
        ./family_mar.exe >>log.txt </dev/null 2>&1 &
        # count how many processes were started in parallel
        let running++
        echo "Check $fam started"
      else
        # handle error case: generator have not produced file
        echo "no input $?"  >>log.txt
      fi
    else
      # hande error case: generator failed
      echo "generator error $?"  >>log.txt
    fi
    cd ../..
    # Check how many processes are running. If there are already 16,
    # wait for one to finish before starting next.
    if [ $running -ge 16 ]; then
      wait -n
    fi
  done

  # There are no more families to check, but some checks are still running. Wait for them to finish.
  echo "Batch complete, waiting for unfinished jobs"
  wait

  echo "Collecting results to results_$1.txt"
  >results_$fam.txt
  for fam in "${families[@]}"; do
    dir=wu_$1/$fam
    md5sum $dir/output.txt >>$dir/log.txt
    echo "# Generator=generator_lk_4_31_31 family=$fam" >>results_$1.txt
    # For each checked family, copy the results into results_$1.txt file.
    cat $dir/output.txt >>results_$1.txt
    cat $dir/log.txt >&4
  done
  echo "Batch complete."
}


function compile()
{
  # Function to compile family_mar from source.
  # The source must be already in build/ directory
  echo "Compiling"
  cd build
  md5sum *.cpp *.h >&4
  g++ -c -O3 -o prov_blk_trans.o prov_blk_trans.cpp >&4 2>&1 || return 1
  g++ -c -O3 -o kanonizator.o kanonizator.cpp >&4 2>&1 || return 1
  g++ -fpermissive -O3 -o family_mar.exe kanonizator.o prov_blk_trans.o prov_blk_main.cpp >&4 2>&1 || return 1
  cd ..
  return 0
}

if ! compile; then
  echo "Compile error"
  exit
fi

md5sum generator_lk_4_31_31.exe hash_tabl.bin >&4

gencheck $1

date >&4
ID: 452 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 453 - Posted: 19 Dec 2018, 10:28:44 UTC

For everyone

I figured out how we will do it.

1. Anyone wishing to receive a block of tasks, please download the archive by the link.
https://cloud.mail.ru/public/Ew8c/HqXB7mpW5

2. Blocks of tasks will be selected from the file names_44323p_21332.txt

This file contains 21332 family names.
These families are the blocks of tasks.

3. Next, you read the instructions in the message
https://boinc.multi-pool.info/latinsquares/forum_thread.php?id=57&postid=430

4. You can write your own script to generate and check the WU.

5. You can use the script by Tomas Brada.

So, everyone should write which families from the names_44323p_21332.txt file he chose for the tasks.
You can choose any number of families, but you don’t need to take a lot of families at once so that the check does not take a long time.

Tomas Brada has already announced which families he chose for the first assignment.

I think that together you will perform this experiment quickly.


What are the questions?
ID: 453 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 454 - Posted: 19 Dec 2018, 10:33:32 UTC - in response to Message 451.  
Last modified: 19 Dec 2018, 10:34:12 UTC

Now you need to check the names_44323p_21332.txt file.
All families are different and do not give the same tasks.

Thank you. I queued up IAEDFXNKG..IAFECTKVH, estimated time 10 hours.

OK.

So, in work is
block1 - IAEDFXNKG..IAFECTKVH

Each selects the next block and reports this.
ID: 454 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 455 - Posted: 19 Dec 2018, 10:36:04 UTC

Tomas Brada
many thanks for your script.
ID: 455 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 457 - Posted: 19 Dec 2018, 10:43:01 UTC - in response to Message 451.  

I queued up IAEDFXNKG..IAFECTKVH, estimated time 10 hours.

Tomas Brada
I see that IAEDFXNKG..IAFECTKVH it is 146 - 1553 families.
Why didn't you choose families 1 - 145?
ID: 457 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tomas Brada

Send message
Joined: 22 Jan 18
Posts: 63
Credit: 373,915
RAC: 0
Message 458 - Posted: 19 Dec 2018, 10:53:59 UTC

I submitted:
IAEDCLHQS..IAEDCRKUH first batch of 17 was run as a test
IAEDCRTKH..IAEDFXKNG second batch i selected 128, because I did not know how long it is going to take.

Now I am running IAEDFXNKG..IAFECTKVH, 1408 tasks. One task takes on average 418 seconds. The generator takes about 3 seconds. I am running 16 in parallel.
ID: 458 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 459 - Posted: 19 Dec 2018, 10:56:50 UTC

Tomas Brada
I have already received results from you
results_a00.txt
results_a01.txt

Thank you!

Now I will process these results.
ID: 459 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 460 - Posted: 19 Dec 2018, 12:17:46 UTC - in response to Message 459.  

Tomas Brada
I processed the results of these blocks
results_a00.txt
results_a01.txt

Найдено марьяжных КФ:
count[1] = 137
count[2] = 104
count[3] = 1
count[4] = 1
Всего: 243
Найдено соквадратов: 352
КФ соквадратов: 243

All is well.
ID: 460 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 461 - Posted: 19 Dec 2018, 15:28:12 UTC
Last modified: 19 Dec 2018, 15:29:02 UTC

jozef j, Peppernrino
it seems you did not understand me.

I repeat: we will perform an experiment with self-generation and verification of WU.

If you want to continue participating in the subproject, please read the topic carefully.

You choose the block of tasks yourself.
Then generate and check the WU yourself.
So does Tomas Brada.

WUs generation and forwarding is extremely tedious.
I give everyone a generator and you can generate WU yourself and then check them.
ID: 461 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jozef j

Send message
Joined: 10 Nov 17
Posts: 10
Credit: 1,168,799
RAC: 0
Message 462 - Posted: 19 Dec 2018, 15:48:28 UTC

I need know what exactly do and checking now other people.. for not do same work as other, in same time
last work i give was block 14 from automated scripts, and this is best way to do in future becouse
for me is waste time do that small scripts/task - i need some big for 64 core cpu,run one time and load on 100% cpu for long time
becouse i run other boinc projects and i can turn them off and run on 100% this autonomous subproject
ID: 462 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 463 - Posted: 19 Dec 2018, 16:17:09 UTC - in response to Message 462.  

jozef j
please read everything carefully, starting with this message.
https://boinc.multi-pool.info/latinsquares/forum_thread.php?id=57&postid=430#430

I think I should not repeat this.

You choose a block of families of any size, but to cope with calculations in a reasonable time.

See how chose the block of tasks Tomas Brada.

See also the script from Tomas Brada.

You can write your own script.
ID: 463 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 464 - Posted: 19 Dec 2018, 17:01:31 UTC

I will show you an illustration of an interesting solution that I found when checking a group of families

AAAATJBQG
AAAATJEDG
AAAATJEDO
AAAATJENA
AAAATJENI
AAAATJHFC
AAAATJHFO
AAAATJHQA
AAAATJHQC
AAAATJKDA


ID: 464 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 220
Credit: 59,067
RAC: 11
Message 465 - Posted: 19 Dec 2018, 17:29:57 UTC - in response to Message 377.  

[
Have you tested 15.967 families in an experiment of Belyshev?
Can you send me the results?


Result sent.
ID: 465 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 467 - Posted: 19 Dec 2018, 18:11:00 UTC - in response to Message 465.  

ice00
I got results.
But ... which families did you check?
They do not follow in a row?

I will process your results tomorrow.
ID: 467 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Peppernrino
Avatar

Send message
Joined: 15 Jan 18
Posts: 13
Credit: 104,273
RAC: 0
Message 469 - Posted: 19 Dec 2018, 18:28:47 UTC - in response to Message 461.  

Natalia I do understand you! I just haven't checked the forum again until now. :P

I would love to participate. Just waking up to my first coffee here, and I'll read the thread more thoroughly and get started ASAP. :D
ID: 469 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 470 - Posted: 20 Dec 2018, 6:38:01 UTC

Tomas Brada
I got the results of block a02.
Thanks!

Now I will process the results.
ID: 470 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 471 - Posted: 20 Dec 2018, 7:44:53 UTC
Last modified: 20 Dec 2018, 8:24:13 UTC

Tomas Brada
I processed your results.
Found 422 unique CF ODLS, including two groups of four ODLS pairs

 0 1 2 3 4 5 6 7 8 9
 1 2 3 0 5 9 7 8 4 6
 2 3 9 5 0 4 8 6 7 1
 5 9 1 6 2 7 0 4 3 8
 6 8 0 4 7 1 3 9 5 2
 9 5 4 7 3 8 2 1 6 0
 3 0 8 2 1 6 4 5 9 7
 7 6 5 8 9 0 1 3 2 4
 4 7 6 9 8 2 5 0 1 3
 8 4 7 1 6 3 9 2 0 5

 0 1 2 3 4 5 6 7 8 9
 1 2 3 4 0 6 9 5 7 8
 4 9 5 2 1 7 8 3 0 6
 9 6 4 7 3 8 0 1 5 2
 7 3 8 1 9 2 5 6 4 0
 2 5 0 6 8 3 7 4 9 1
 8 0 6 5 2 4 1 9 3 7
 5 7 1 9 6 0 4 8 2 3
 3 4 9 8 7 1 2 0 6 5
 6 8 7 0 5 9 3 2 1 4

All is well.

Do you continue?
ID: 471 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 15 · Next

Message boards : News : Autonomous Subproject

©2024 ©2024 Progger & Stefano Tognon (ice00) & Reese