please choosego to mobile | Continue to access the PC version
View: 24490|Reply: 17

OPI One Cluster Processing

[Copy link]

1

threads

6

posts

122

credits

Registered member

Rank: 2

credits
122
Published in 2016-5-1 21:00:50 | Show all floors |Read mode
Edited by jadr2ddude at 2016-5-1 21:01

I am planning a project to experiment with the use of Orange Pi’s for cluster processing. This will be my sophomore research project for High Technology High School. My goal is to provide speed to an Intel server at much lower cost by using many Orange Pi Ones. When my project is complete, I will release a system for easily setting up, deploying, managing, and utilizing an Orange Pi cluster. Here are some possible tasks it could be used to speed up:
  • Compilation (distcc)
  • Web Serving
  • Dynamic Web Resource Generation
  • Distributed Neural Networks
  • Simulations
  • Storage Systems
This project will use a buildroot system to provide containerization and will also manage the networking. I will provide the compiled image, config, and patches for the buildroot system.

Details
I plan to use the USB OTG ports on Orange Pi One SBCs for networking. They will all connect to a USB 3.0 hub which will then connect to a router running OpenWRT. Because they are USB 2.0, each Orange Pi One will be able to transfer data at up to 450 megabits per second – 4.5 times faster than the Ethernet ports. The hub will also have a SSD connected via a SATA-to-USB adapter. Because USB 3.0 can carry 5Gbit/second, all of the SBCs can communicate at top speed simultaneously, but not with the SSD running at top speed at the same time. This is still plenty of routing power for most purposes.
Scalability
There will be another set of six OPIs plugged into the first set (total of twelve boards in my cluster). This can be repeated to add more computing power, but will have declining returns due to increasing latency and decreasing bandwidth per node. To further expand the cluster, the OpenWRT router can run a software defined network.

Single choicepoll, A total of people voted
91.30% (21)
0.00% (0)
8.70% (2)
Your usergroup does not have voting rights

1

threads

6

posts

122

credits

Registered member

Rank: 2

credits
122
 Author| Published in 2016-5-2 00:40:19 | Show all floors
I will be using the g_ether driver.
Reply

Use magic report

1

threads

6

posts

122

credits

Registered member

Rank: 2

credits
122
 Author| Published in 2016-5-2 06:42:03 | Show all floors
trpaslikcz replied at 2016-5-2 02:37
Interesting. This way each SBC will appear as an USB network dongle to a connected host. Did you a ...

The SSD will be mounted on the OpenWRT router and the filesystem will be shared to the nodes. I will be testing the performance and CPU usage of the networking once I have the images compiled.
Reply

Use magic report

1

threads

6

posts

122

credits

Registered member

Rank: 2

credits
122
 Author| Published in 2016-5-3 03:06:18 | Show all floors
Then I will write a new driver/protocol.
Reply

Use magic report

11

threads

75

posts

376

credits

Intermediate member

Rank: 3Rank: 3

credits
376
Published in 2016-6-13 08:21:54 | Show all floors
interesting theme.

a easiest way to build cluster/grid/botnet/skynet:

download and build inferno-os. it is a framework/system designed especially to develop distributed networks ( https://en.wikipedia.org/wiki/Inferno_(operating_system) ). it can run as framework on win/lin/mac OSes, x86/arm/sparc/raspberypi like and many others cpus. and it can run as OS itself. yes, it's a oldschool skynet software.

here the main free sources repo - https://bitbucket.org/inferno-os/inferno-os/

it's easy to buid, but here is a prebuilt package (inferno-owen.xz) - https://mega.nz/#F!ZshHFJ5J!8-TKqn9gTKZh6QybGKiAjg
unpack somewhere, go to inferno-os folder, run emu.sh

cluster/grid/botnet/shwarz.. hm.. others (easyest kind):

1) one (only one) of nodes (computers in the grid) should contain the registry of currently connected nodes. its IP address should be written to

*inferno-os/lib/ndb/local*

line:
```
infernosite=
...
...
   registry=<IP address of registry node>
```
on every node in your botnet.

2) you need start emu on the registry node, run shell (if it started in the graphical mode), and type in the shell:
```
ndb/cs #start the connection server
mount -A -c {ndb/registry} /mnt/registry #mount registry server to /mnt/registry (-A - without security)
listen -A -v 'tcp!*!registry' {export /mnt/registry&} #listen to network and export mounted registry service to just connected node
```

3) start emus on worker nodes and type in their shells:
```
ndb/cs
mount -A 'net!$registry!registry' /mnt/registry  #connect to registry node and mount exported registry
#the should register itself in the registry
myip=`{os hostname -I} #get self IP (command "os" runs the command/program of hosting OS, "hostname -I" is here)
grid/reglisten -A -r svc rstyx 'tcp!' ^ $myip ^ '!5678' {auxi/rstyxd&} #register node, run styx (network protocol) server, listen network for rcmd and export rstyxd. very complex command
```
initialisation has finished (all without security - -A key). howto use:

you can see all connected nodes from any connected node:
```
ls /mnt/registry
```

you can send a job to a node:
```
rcmd -A tcp!<IP of a worker node>!5678 <command to execute remotelly. it may contain os cmd>
```

you can send a job to all connected nodes:
```
for i in `{ndb/regquery -n svc rstyx} {
    rcmd -A $i <your command>
}
```

in the inferno-owen.xz has file gridlib with this examples written. to use it:
in the shell on registry node:
```
run gridlib
masterinit
```

on worker nodes:
```
run gridlib
workerinit
newcpu
```

now you can use rcmd (or rsplit to run a cmd on all connected nodes)

this was a simple, tutorial example
Reply

Use magic report

11

threads

75

posts

376

credits

Intermediate member

Rank: 3Rank: 3

credits
376
Published in 2016-6-14 17:47:31 | Show all floors
..continue

how to exchange of data?

first, in inferno the file tree is a name tree as it contain all what you want and can be changed for each thread. as example 2 threads of single program can use completely different name trees where same paths will mean different files, devices, channels etc

when **rcmd** is called it sends the name (file) tree of thread-caller to remote thread (this communication, connection and name trees are local to pair of connected threads. other pc/threads doesn't see them). remote thread can access to name tree of thread-caller by path */n/client*
```
rcmd -A tcp!<IP address of remote pc already connected to this grid>!5678 ls /n/client  #you can see the file/name tree of thread-caller but sent by remote thread
```

how to create a named pipe mapped to name tree to communicate with remote
```
mkdir /tmp/my_pipe  #this folder will be the name of pipe
bind "#|" /tmp/my_pipe  # '#|' - is a internal pipe driver (all internal driver names of inferno starts from #)
                        # you can see them using *cat /dev/drivers
```
a named pipe with name */tmp/my_pype* was created. you can see it
```
ls /tmp/my_pipe  # you will see files/names *data* and *data1*
                 # data is the input of this pipe
                 # data1 is the output of this pipe
```

let's try to use just created pipe
```
rcmd -A tcp!<IP of remote>!5678 cat /n/client/tmp/my_pipe/data1 & #read the output of the pipe on remote pc
echo hello remote! > /tmp/my_pipe/data  # send a message via our pipe to remote pc
```
that's all for today.
Reply

Use magic report

1

threads

6

posts

122

credits

Registered member

Rank: 2

credits
122
 Author| Published in 2016-6-19 21:39:31 | Show all floors
That sounds good! I may try Inferno afterwards. My main plan is to use docker, because most of the software I will be testing with will work easily with just Linux (e.g. LAMMPS).
Reply

Use magic report

11

threads

75

posts

376

credits

Intermediate member

Rank: 3Rank: 3

credits
376
Published in 2016-6-24 09:31:37 | Show all floors
@jadr2ddude
you can use linux, but inferno (as plan9, as qnx, they all used same ideas) was designed especially for building systems distributed by network.
where many computers from different places work as single computer.
inferno and plan9 designed by bell labs, the father of unix, C, C++ and other
these were military projects (yes, it was not a joke when i named it skynet. it was the real development and inferno planned to be a core of real network of military things (rockets, tanks, avions, your army knife))
because of military order they were closed long time.

inferno was designed for highest portability (for example it can run on top of other OS as framework or without OS on bare hardware)
and easy programming. it has 2 languages interpreted sh (my examples here in sh) and compiled limbo (it looks like something between C++, pascal/oberon with some very highlevel constructions like from python, and it has easy to use multithreading system with mutexes, channels etc embedded directly into the language (as go or xc))
it also includes graphical sourcelevel debugger like ddd (also was writen in limbo and can work without recompilation)

it can be compiled under linux (and windows and mac and other) in minutes.
it has as minimum 3 pluses
1. you can develop your program on any system you want and run on other system without recompilation as it is a virtual machine
2. it is significally faster then other VMs including java and .net because of internal architecture, and because of jit compilers for most of architectures (x86, arm, mips, sparc, power)
3. it takes less of resources, thanks to very good garbage collector (when i tested it on win ce it took only 7-8mb with gui environment and pair of programs runned)

also it provides many abilities now absent in regular OSes or programming languages (maybe excluding go - a son of limbo, and xmos xc - also looks like limbo)


a example - how to browse/parse webpages directly from shell:
```
ndb/cs   #start connection service

fn rd_foo { #foo to read a page
  id=$1

  {
    echo 'GET /orangepibbsen/forum.php?mod=viewthread&tid=1459&extra=page%3D1 HTTP/1.1' &&
    echo 'connection: close' &&
    echo 'host: www.orangepi.org' &&
    echo ''
  } > /net/tcp/$id/data  #send a regular GET query to server

  cat /net/tcp/$id/data  #output received from server page to terminal

#  var=`{read < /net/tcp/$id/data} #or catch it to variable for parsing as example
}

fn get_opi_forum {  #foo to connect to a server
  (null ip)=`{ndb/csquery net!www.orangepi.org!http}  #get the IP of the server
#  echo $ip  # you can see received IP

  {
    id=`{read}  #get the id of just opened tcp channel
    echo connect $ip > /net/tcp/$id/ctl  #connect tcp channel $id to your server

    rd_foo $id #run the foo to read a page
  } <> /net/tcp/clone #by opening this file new tcp channel creates, by closing closes
                      # <> - mean - open for read and write and no close till the end
                      # of all commands between {}
}

get_opi_forum  #do the main foo
```
(all examples till now were in sh. inferno allows to make much operations with sh then linuxes' bash, for example regular replace, building of GUIs)
(limbo runs faster then sh but it will be in the future)

you can call commands of hosting OS from sh using command "os" and send/receive data from it to inferno
```
os <command string of hosting os>
```

next time i try to write how write a extension to inferno callable as limbo module in C (inferno contain a special tool to help in writing of C extensions)
Reply

Use magic report

11

threads

75

posts

376

credits

Intermediate member

Rank: 3Rank: 3

credits
376
Published in 2016-6-24 09:55:22 | Show all floors
PS i forgot. here is a ready to use inferno based distributed system
https://bitbucket.org/inferno-os/owen

some slides about inferno-owen:
http://www.resc.rdg.ac.uk/presentations/Inferno_25-03-04/
www.vitanuova.com/papers/ugrid.pdf

it already included in the package from my mega.nz. if you interested, i try to write a pair of examples how to use it
Reply

Use magic report

1

threads

6

posts

122

credits

Registered member

Rank: 2

credits
122
 Author| Published in 2016-6-24 10:54:56 | Show all floors
Yes I will consider also trying out inferno, but most of the stuff I need to run/test is in C/C++.
Reply

Use magic report

You need to log in before you can reply login | Register

Points Rule

Quick reply Top Return list