Nabble - PelicanHPC

KestrelHPC GNU/Linux

2009-11-25T00:48:38Z

KestrelHPC (http://www.ceit.es/mechanics/people/cv/dborro/Students/kestrelHPC-web/kestrelHPC/KestrelHPC.html) is another new system for setting up a HPC cluster, using a frontend machine that runs Debian or Ubuntu GNU/Linux.

It allows for multiple users, whereas PelicanHPC is for a single user. From the web page: "This project offered what we were looking for (easy to use and an easy interface to configure the cluster), yet, we wanted an installed linux instead of a "live" distribution. Hence, on account of to the aforementioned, we started developing a script that would make a Debian or an Ubuntu, a Pelican-like linux. This consists of, a script that installs all the packages needed to have a working Head-node and besides this, copies the image that will be sent to the slave nodes by LAN. Finally, modified pelican scripts and new kestrel scripts are copied to administrate the cluster in a very intuitive way. "

ABC GNU/Linux

2009-11-18T04:47:47Z

http://www.ehu.es/AC/ABC.htm

There is a new distro out which works much like PelicanHPC. It is Ubuntu-based, and it can be installed to disk. I have tried it in live mode, and it works nicely. The same caution that you need to use running PelicanHPC on an open network needs to be applied with this distro - it acts as a dhcp server out of the box, which can interfere with existing networks.

I recommend giving it a try.

Mirrors updated

2009-11-17T05:39:01Z

Hello everyone,

I just updated the mirrors last night.
I just discovered a problem with my University mirror, the speed problems seems to be larger than expected, as the whole network at University here is quite slow... I'm working on it, sorry for this.

Primary (may be around 1MB/s)
http://download.mi.hs-heilbronn.de/pelicanhpc/

Secondary (should run up to 8 MB/s)
http://download.semmel.org/PK/

Both mirrors are located in Germany. I'll update this post everytime a new version occurs and the mirrors are updated.

Thank you,
Simon

Re: Add drivers to parallel-knoppix???

2009-11-16T00:38:24Z

Remastering ParallelKnoppix is quite complicated, compared to customizing PelicanHPC. It uses a self-compiled kernel, and adding modules is a little tricky (plus I have forgotten the details). I seriously recommend using PelicanHPC if at all possible.

Re: Full hdd install??

2009-11-16T00:36:03Z

I have no plans to add this to PelicanHPC, but one could certainly do this. I know of one project that is working to make something similar to PelicanHPC, but for an installation. That project has not yet gone public, though.

It's worth remembering that the PelicanHPC frontend can be run withing a virtualization environment, and that it can be made to use permanent storage for /home. Snapshotting a virtual machine means you can recover the state very quickly. Use of permanent storage for /home means that you can save and recover your work easily. Because these solutions work well, I don't plan on taking on the project of an installable version. For an installable cluster distro, I think that Rocks and Perceus/Warewulf are the leading solutions.

Re: Full hdd install??

2009-11-15T07:46:37Z

Script to make front end on existing system.
This script created the compute nodes, and prepares an existing debian system to serve as a pxe-server, instead of live-cd.
I hope it works fine, I could not test it still. Please give some feedback if you find this erroneous or useful.

After running it, try and boot one of the pxe-nodes, this should work.

###BEGIN SCRIPT#####
#!/bin/sh
############ packages to add - place names of packages you want here ####################
cat <<PACKAGELIST > addlist
# basic stuff needed for cluster setup
ssh dhcp3-server nfs-kernel-server nfs-common atftpd
# binary blobs for networking
firmware-bnx2 firmware-iwlwifi firmware-ralink linux-wlan-ng-firmware
# resource management
slurm-llnl slurm-llnl-sview slurm-llnl-basic-plugins
# configuration and tools
wget bzip2 dialog less net-tools rsync fping screen
make htop fail2ban locales console-common ifenslave
# mail support (to forward status reports)
bsd-mailx liblockfile1 mailx postfix ssl-cert
# MPI
lam-runtime lam4-dev openmpi-bin libopenmpi-dev
# Octave
octave3.0 octave3.0-headers gnuplot
# Python
python-scipy python-matplotlib python-numpy ipython lampython
# other scientific
gfortran libatlas-headers libatlas3gf-base
# X stuff
xorg xfce4 konqueror ksysguard ksysguardd kate
konsole kdenetwork kdeadmin kcontrol kpdf
# other stuff, including requests from users
bc libxp6 vim hal libhal1 libhal-storage1
PACKAGELIST
################## END OF PACKAGELIST ################

PELICAN_NETWORK="10.11.12"
MAXNODES="100"
#ARCHITECTURE="amd64"
#KERNEL="amd64"
ARCHITECTURE="i386"
KERNEL="686"
IMAGETYPE="iso"
#IMAGETYPE="usb-hdd"
DISTRIBUTION="lenny"
MIRROR="de"

# leave the rest of this alone unless you really know what you're up to
THISDIR="`pwd`"
mkdir "$ARCHITECTURE"
cd "$THISDIR/$ARCHITECTURE"

# ############## compute node configuration ##########
# this section should be uncommented the first time
# you make Pelican for an architecture, then commented
# out, to avoid wasting time and bandwidth
LIVECDDIR="nodes"
#rm -R -f "$THISDIR/$ARCHITECTURE/$LIVECDDIR"
rm -R -f "$THISDIR/$ARCHITECTURE"/tftpboot
mkdir "$THISDIR/$ARCHITECTURE/$LIVECDDIR"
cd "$THISDIR/$ARCHITECTURE/$LIVECDDIR"
lh_config \
-a "$ARCHITECTURE" \
--categories "main contrib non-free" \
--packages "firmware-bnx2 firmware-iwlwifi firmware-ralink linux-wlan-ng-firmware" \
-b net \
-d "$DISTRIBUTION" \
-k "$KERNEL" \
--mirror-binary http://ftp.$MIRROR.debian.org/debian/ \
--mirror-chroot http://ftp.$MIRROR.debian.org/debian/ \
--mirror-bootstrap http://ftp.$MIRROR.debian.org/debian/ \
--mirror-binary-security http://security.eu.debian.org/ \
--mirror-chroot-security http://security.eu.debian.org \
--net-root-server "$PELICAN_NETWORK".1 \
--net-root-path "/var/lib/tftboot" \
--bootappend-live "noautologin noxautologin" \
--syslinux-menu enabled \
--syslinux-timeout 5
lh_clean
# use local packages if they're there
rsync -az "$THISDIR"/packages/* config/chroot_local-packages
# chroot hook to use local includes (new aug 2008)
cat <<CHROOTHOOK > config/chroot_local-hooks/script.sh
#!/bin/bash
update-initramfs -u -k all
CHROOTHOOK
chmod +x config/chroot_local-hooks/script.sh
# SYSLINUX SPLASH SCREEN
if [ -e "$THISDIR/splash.rle" ]; then
cp "$THISDIR"/splash.rle config/binary_syslinux/splash.rle
lh_config --syslinux-splash "config/binary_syslinux/splash.rle"
fi
# post boot script to get nodes to talk to frontend
install -d config/chroot_local-includes/usr/share/initramfs-tools/scripts/live-bottom/
cat <<99START > config/chroot_local-includes/usr/share/initramfs-tools/scripts/live-bottom/99script
#!/bin/sh
cat <<RC_LOCAL > /root/etc/rc.local
#! /bin/sh
pelican_boot_setup
RC_LOCAL
chmod a+x /root/etc/rc.local
99START
chmod +x config/chroot_local-includes/usr/share/initramfs-tools/scripts/live-bottom/99script
lh_build
cd "$THISDIR/$ARCHITECTURE"
mv "$THISDIR/$ARCHITECTURE/$LIVECDDIR"/tftpboot "$THISDIR/$ARCHITECTURE"/
# safeguard against crashes
umount proc-live
umount sysfs-live
umount devpts-live
sync

mkdir -p /var/lib/tftboot/
rsync -az "$THISDIR/$ARCHITECTURE"/tftpboot/ /var/lib/tftpboot/

# Remove additional spaces
DV="\$(echo \$NETDEVICES)"
fi
IP="$PELICAN_NETWORK.1"
NM="255.255.255.0"
BC="$PELICAN_NETWORK.255"
ifdown \$DV
sleep 4
CMD="ifconfig \$DV \$IP netmask \$NM broadcast \$BC up"
\$CMD
sleep 4

# configure dhcp
cat <<DHCP > /etc/dhcp3/dhcpd.conf
# global settings
allow booting;
allow bootp;
default-lease-time 600;
max-lease-time 7200;
subnet $PELICAN_NETWORK.0 netmask 255.255.255.0 {
next-server $PELICAN_NETWORK.1;
filename "pxelinux.0";
option subnet-mask 255.255.255.0;
range $PELICAN_NETWORK.2 $PELICAN_NETWORK.$MAXNODES;
}
DHCP

KERNELVERSION=\`uname -r\`
# configure stuff for serving up netboot image this was not needed in live-helper 1.0.3
# but it is needed for the current version. Perhaps can remove in future.
cat << PXEFIX > /var/lib/tftpboot/debian-live/$ARCHITECTURE/pxelinux.cfg/default
prompt 1
timeout 10
default live
LABEL live
MENU LABEL Start PelicanHPC compute node
kernel debian-live/$ARCHITECTURE/vmlinuz-\$KERNELVERSION
append initrd=debian-live/$ARCHITECTURE/initrd.img-\$KERNELVERSION boot=live noautologin noxautologin union=aufs netboot=nfs nfsroot=10.11.12.1:/live/image
PXEFIX

# tftpd - use one or the other
# configure tftpd-hpa
cat <<TFTP > /etc/default/tftpd-hpa
#Defaults for tftpd-hpa
RUN_DAEMON="yes"
OPTIONS="-l -s /var/lib/tftpboot"
TFTP

# generate /etc/exports
cat <<EXPORTS > /etc/exports
/var/lib/tftpboot *(ro,async,no_subtree_check,no_root_squash,fsid=12345)
/home $PELICAN_NETWORK.0/255.255.255.0(rw,root_squash,async,no_subtree_check)
EXPORTS

ifconfig eth0 10.11.12.1
/etc/init.d/tftpd-hpa start
/etc/init.d/dhcp3-server start
/etc/init.d/nfs-kernel-server start

Re: Full hdd install??

2009-11-15T07:23:44Z

Hi, I'm also very much interested in a solution like this.
I've started modifying the make_pelican for this purpose.

Basically, what you need to do is to create the live-cd's for the nodes, and configure an existing debian system to serve as a PXE server.

I'll post my modified script when I'm done.

Full hdd install??

2009-11-14T20:46:49Z

Hi

in future releases will pelicanHPC have the ability to fully install to a HDD??

so i would be able to install apps and save settings and get rid or Fxce and put kde or Gnome on??

thanks

Add drivers to parallel-knoppix???

2009-11-13T10:10:00Z

Hi

Just wondering how to add new network drivers for the pxe boot on parallel-knoppix
pelican hpc has the right drivers for my eeepc and i want to add them to parallel-knoppix

any ideas????

Thanks

Re: networking

2009-10-30T23:26:07Z

Often, the dhcp client will take care of that. To do it manually, you can use kcontrol for a graphical frontend, or you can edit the file /etc/network/interfaces to set things the way you like. For setting up the netorking of the cluster, the pelican_setup script will do it for you.

networking

2009-10-28T15:28:58Z

Hello,
I'm new to the Debian flavor of Linux and haven't had much success in determining how to configure networking (setting the default gateway, ip address for one card, etc.).
Thanks,
-Tom

Re: newbie question

2009-10-19T08:59:28Z

No, it's not that simple, unfortunately. For parallelization using MPI, the source code has to include specific instructions. There are some compilers that can automatically parallelize certain types of code, but that is a different approach from what PelicanHPC does. Such compilers usually cost a good deal of money. To get a benefit from PelicanHPC, the application has to use MPI.

Re: newbie question

2009-10-19T04:52:29Z

Hi, as I am also a newbie and interested in the same thing, I'll post in here.

I run model simulations with Modelica, and as for some models it takes a lot of time I thought of using Pelican.

There is a linux version of Modelica, but I think it won't get parallelized.
Modelica is a language for modeling purposes, and to evaluate the models it compiles into C code and then executes.

I've been reading a lot in this forum, and, I have one question:
As modelica - openmodelica- is opensource, if I compile it somehow using mpicc or similar, it will get paralellized?

Thank you!

Re: starting problems with Pelican

2009-10-02T06:28:04Z

you,re right...

Timing results, second example
Number of nodes: 0 Time (sec.): 626.631167
Timing results, second example
Number of nodes: 1 Time (sec.): 311.761412
Timing results, second example
Number of nodes: 2 Time (sec.): 171.927231
Timing results, second example
Number of nodes: 3 Time (sec.): 113.922598
Timing results, second example
Number of nodes: 4 Time (sec.): 92.087825
Timing results, second example
Number of nodes: 5 Time (sec.): 73.063092
Timing results, second example
Number of nodes: 6 Time (sec.): 73.621497
Timing results, second example
Number of nodes: 7 Time (sec.): 54.708440
Timing results, second example
Number of nodes: 8 Time (sec.): 54.226124
octave:34>

It,s working great... thank you again for your help. PelicanHPC is a really nice linux distro to learn something about clusters. Now it,s working i,m going to read a little bit about MPI and try to start RAxML on the cluster.

Re: starting problems with Pelican

2009-10-02T05:41:19Z

Your cluster should do better than that for Monte Carlo. Try the code below. This uses a large number of reps, so that the compute nodes are busy for a little while. Also, the nodes do 1000 replications at a time, rather than 100, to lower network effects. Finally, and importantly, I moved "toc;" to avoid counting the time used to make the bar graph (stupid bug!). I think this last bug is what caused the problem.

########### cut here ###############

# this runs the test on several different configurations and reports timings.
# to see good speedups in parallel, you need to set "reps" to a fairly
# big number (say, 100000 or so).
outfile = "example2.out";
T = 1000;
dim = 5;
reps = 100000;

maxslaves = 8;
timings = zeros(maxslaves,1);
for i = 0:maxslaves;
# figure;
tic;
n_received = montecarlo("tracetest", {T,dim}, reps, outfile, i, 1000);
t = toc;
if i > 0
bar(0:i, n_received/reps);
xlabel("node");
grid on;
title("percentage of results received by node");
drawnow();
endif

timings(i+1,:) = t;
endfor

for i = 0:maxslaves;
printf("\n\nTiming results, second example\n");
printf("Number of nodes: %d Time (sec.): %f\n", i, timings(i+1,:));
endfor

Re: starting problems with Pelican

2009-10-02T05:28:13Z

We're hosting 9 similar machines, it was a gift from a offical company, all have the same mothoerboard and CPU and RAM. The montecarlo example 2 is nice... I think, maxslaves is the number of maximum nodes that should be used? So if I increase the number of reps to 10,000 - the time is going down for the first 2 nodes and then it stucks there, later the time increases also, like:

0 node - 65 sec
1 node - 45 sec
2 node - 25 sec
3 node - 25 sec
...
8 node - 26 sec

So this is a problem of our slow ethernet? The mobos have only a fast and the switch is a 100mbps full duplex switch. I read in the forum someting about benchmarking, but if I'm honest, I didn't understand how to optimize it. I'm now a little confused if what I'm trying is actually working - to compute my phyolgeny stuff over night instead of 1 week...

Re: starting problems with Pelican

2009-10-02T05:04:50Z

OK, I'm glad it's working. I guess that the node that has CPU usage at 90% has a slower CPU than those of the other nodes. They are mostly idle because they are waiting for the slow node to finish. The speedup you get depends on the relative speeds of the nodes, and on the latency and bandwidth of the networking that connects them. The results you obtain depend very much on the details of the cluster. Doing work on a single fast computer will often be faster than using a 2 node cluster made of the fast computer plus a slow node, unless you have load balancing built into the code. This is because if you just give the slow computer half of the computations, the power of the fast computer will not be fully used.

The pea_example code is fairly "tightly coupled" which means that it is sensitive to the latency and bandwidth of the network.

I have a paper out that discusses this example, at
http://www.springerlink.com/content/b2l43gg723781m87/

There are also some Monte Carlo examples using Octave that are not so sensitive to the networking hardware. Try running mc_example1 and mc_example2 while in Octave.

Re: starting problems with Pelican

2009-10-02T04:47:01Z

So, I edited the pea example to the count of nodes we do have and it's working. All nodes run a command "octave". funnily enough the cpu load of the nodes just lays between 25 - 30% - only node 1 has a cpu load over 90% - why come so? And I thought, the calculation is faster, like if you had 10 seconds for 60 iterations with 2 nodes, it's only 5 or someting with 8 nodes... but it isn't.

Re: starting problems with Pelican

2009-10-02T03:25:47Z

Thank you, I really appreciate your work and support. I'll try this and give you a feedback. We only tried to edit the kernel_example.

Re: starting problems with Pelican

2009-10-02T03:03:58Z

No, the examples don't necessarily use all available nodes, the number is often hardcoded. With pea_example, you can change the number of nodes by calling "edit pea_example" from inside octave, and increasing the number of compute nodes in the line

nodes = 1; # number of compute nodes to use (0 for serial)

If you increase that number, then you should see activity on more nodes.

Cheers, M.

Re: starting problems with Pelican

2009-10-02T02:53:39Z

Michael Creel wrote:

Well, it seems that things worked after the restart. Now that I think about it, this is a circumstance that does occur occasionally - if you click yes while a compute node is just finishing booting, sometimes you see what has happened to you. The solution is to just re-run pelican_restart_hpc. So this is normal. The way to never see this problem is to wait a bit longer before clicking yes, so that the compute nodes all finish booting. Perhaps I could add a little sleep time to the script to avoid this.

In general, I don't recommend using v1.99.0, unless you need the newer Linux kernel, or unless you really want to use kernel regression or density smoothing in Octave. v1.9.1 has had a lot more testing. I believe that newer Semprons are 64 bit, so it should work, if you decide to try it. It depends on whether or not the CPUs are 64 bit.

If 2 MPI ranks are used, it is normal that top only shows 1 running on the frontend, because the other is running on a compute node. To see it, ssh into the compute node and run top there (or set up a cluster monitor - see the homepage for a link how to do it)

Ok, I remember: somewhere it's written like "better use 1.9.1" - and actually I'm impressed how easy it was to setup a cluster with pelican. Now the only concern I do have is that not all nodes are computing. So I did what you told me and logged me into the nodes and run "top" - but only 1 node showed octave at the pea_example. I can ping and ssh the nodes. Interestingly sometimes if I start a example, the failban2server appears in top. Actually all examples should use all achievable nodes?

Re: starting problems with Pelican

2009-10-02T02:06:45Z

Well, it seems that things worked after the restart. Now that I think about it, this is a circumstance that does occur occasionally - if you click yes while a compute node is just finishing booting, sometimes you see what has happened to you. The solution is to just re-run pelican_restart_hpc. So this is normal. The way to never see this problem is to wait a bit longer before clicking yes, so that the compute nodes all finish booting. Perhaps I could add a little sleep time to the script to avoid this.

In general, I don't recommend using v1.99.0, unless you need the newer Linux kernel, or unless you really want to use kernel regression or density smoothing in Octave. v1.9.1 has had a lot more testing. I believe that newer Semprons are 64 bit, so it should work, if you decide to try it. It depends on whether or not the CPUs are 64 bit.

PelicanHPC should let you use any MPI programs that work with LAM/MPI or OpenMPI. It's main purpose is precisely to provide an easy to set up platform for working with MPI. I think that the code you mention should work fine, but I have no experience with it.

Cheers, M.

Re: starting problems with Pelican

2009-10-02T01:48:05Z

Hello Michael, thank you for the fast answer...

Michael Creel wrote:

So, the setup process goes as it appears in the Tutorial or the screencast, and seems to complete normally. The setup process reports finding the compute nodes, as in this shot:
but when you finish the setup process and call "lamnodes" is says that lamd is not running? Is that an accurate summary of the problem?

That's correct. The screens are the same like in the turtorial and at the end the screen says: Found 8 nodes (we're running 8 at the moment)

Michael Creel wrote:

What version of PelicanHPC are you using?

From this mirror: http://download.mi.hs-heilbronn.de/pelicanhpc/
pelicanhpc-v1.9.1-32bit.iso

Michael Creel wrote:

After you re-run pelican_restarthpc things seem to work correctly?

Interestingly it's "pelican_restart_hpc" (with underscore), but yes & no, I did the following:
1.) lamboot
2.) lamnodes shows only the host
3.) pelican_restart_hpc
4.) the host finds 8 nodes, but lamnodes has still only 1
5.) pelican_setup again
6.) the host finds 8 nodes, lamnodes shows all 8 IPs of the nodes
7.) startx and then I tried some examples

Michael Creel wrote:

kernel_example has a bug, thanks for the report. I'll fix that for the next release. (EDIT: actually, this is already fixed in the v1.99.0 release.). I think that pea_example is running fine. This is an indication that re-running pelican_restarthpc in fact worked. If 2 MPI ranks are used, it is normal that top only shows 1 running on the frontend, because the other is running on a compute node. To see it, ssh into the compute node and run top there (or set up a cluster monitor - see the homepage for a link how to do it).

Ok, here we go: v1.99 - We do have AMD Semprons 3000+ - and i'm not sure if they support 64bit - so maybe here settles the problem we do have? We tried running the v1.9.1-32bit.iso as I mentioned.

Michael Creel wrote:

sorry, I don't know anything about that.

No problem, but actually MPI Programs should be supported by Pelican? I couldn't find any "howto" for MPI Porgrams to run it on al cluster... it's quite new for me... maybe you have a hint.

Re: starting problems with Pelican

2009-10-02T01:13:35Z

meckart wrote:

wie tried to setup a cluster with Pelican hpc as in the tutroial. The head is running fine, the nodes are booting over lan. The setup script is finding the nodes, so we are pushing "yes" - but then if we execute "lamnodes" it says: lamd is not running, exceute lamboot or something... so we start lamboot, the pelican_restart_hpc, search for nodes and the lamnodes shows the 10.11.12.* - so we are thinking: ready to run.

So, the setup process goes as it appears in the Tutorial or the screencast, and seems to complete normally. The setup process reports finding the compute nodes, as in this shot:

but when you finish the setup process and call "lamnodes" is says that lamd is not running? Is that an accurate summary of the problem?

What version of PelicanHPC are you using?

After you re-run pelican_restarthpc things seem to work correctly?

meckart wrote:

Now the problems: if we're running "kernel_example" we've got errors in octave. If we're running "pea_example" it runs the example, but only with 2 nodes (it says 2 nodes in ocatve and if we're watching the nodes we find only in one top "octave".

So, what are we doing wrong? We stepped through the tutorial several times.

kernel_example has a bug, thanks for the report. I'll fix that for the next release. (EDIT: actually, this is already fixed in the v1.99.0 release.). I think that pea_example is running fine. This is an indication that re-running pelican_restarthpc in fact worked. If 2 MPI ranks are used, it is normal that top only shows 1 running on the frontend, because the other is running on a compute node. To see it, ssh into the compute node and run top there (or set up a cluster monitor - see the homepage for a link how to do it).

meckart wrote:

At the end we want to try · RAxML-VI-HPC (version 2.2.3)
http://icwww.epfl.ch/~stamatak/index-Dateien/Page443.htm

So, we figured out to make the executing - but it is only running on the head...
If sombody wants to help us, feel free :)

sorry, I don't know anything about that.

starting problems with Pelican

2009-10-01T06:43:00Z

Hello,

wie tried to setup a cluster with Pelican hpc as in the tutroial. The head is running fine, the nodes are booting over lan. The setup script is finding the nodes, so we are pushing "yes" - but then if we execute "lamnodes" it says: lamd is not running, exceute lamboot or something... so we start lamboot, the pelican_restart_hpc, search for nodes and the lamnodes shows the 10.11.12.* - so we are thinking: ready to run.

Now the problems: if we're running "kernel_example" we've got errors in octave. If we're running "pea_example" it runs the example, but only with 2 nodes (it says 2 nodes in ocatve and if we're watching the nodes we find only in one top "octave".

So, what are we doing wrong? We stepped through the tutorial several times.

At the end we want to try · RAxML-VI-HPC (version 2.2.3)
http://icwww.epfl.ch/~stamatak/index-Dateien/Page443.htm

So, we figured out to make the executing - but it is only running on the head...
If sombody wants to help us, feel free :)

Martin
University of Jena

Re: help me about creating cluster by pelican hpc iam beginer

2009-09-29T12:18:12Z

Often times to set up PXE boot you need to enable it in a few places in the bios. On the computer I've used Pelican on I needed to enable PXE boot on the ethernet card, enable the bios to boot from PXE and then set PXE to the first boot device...or some similar procedure...I haven't messed with it in a while, but if you hunt around in your bios long enough you should be able to figure it out. Hope this helps.

Re: Remaster with external storage

2009-09-27T06:37:02Z

Probably not, as it's intended for use by a single user. It runs well on hardware that often is used for servers, though. It also runs fine virtualized.

Re: Remaster with external storage

2009-09-26T08:48:03Z

Cool thanks.

Forgot to ask is Pelican suitable for server use?

Re: Remaster with external storage

2009-09-25T23:07:28Z

If a reboot occurs (power goes out, whatever) you won't loose whatever is on /home, if it's on permanent storage. You just need to specify its location again, the same as when you did the first time. The setup script will wait for you to do so. For this reason, I don't see any need to make a remaster that mounts a specific device. If you want to do so, search the make_pelican script for

mount /dev/\$HOMELOCATION /home

and replace $HOMELOCATION with the device name you want to use. That should do it. You might also want to comment out the dialog that asks you to input the home location device.

Remaster with external storage

2009-09-25T07:17:19Z

Hi, I've got a room full of old machines that I'm wanting to try setting up with Pelican. I see its possible to move the /home directory to an external source (ie HD, external HD etc) but can you remaster the cd so that it always points to that location? I'd like this to be as hands off as possible and if it reboots I'd hate to lose my way /home :p

Re: HOW TO: For those whose nodes don't support PXE booting

2009-09-23T16:05:46Z

Hi I found PXE Boot Images Generated on this
http://rom-o-matic.net/

An easy way to get PXE Boot Images for almost every use and Hardware.

Output Formats available:
- Floppy bootable ROM Image .dsk
- Binary ROM Image .rom
- ISO bootable image .iso
- ISO bootable image with legacy floppy emulation .liso
- LILO/GRUB/SYSLINUX loadable Linux kernel format .lkrn
- PXE bootstrap loader format ROM Image .pxe
- USB Keychain Disk Image / bootable HD image .usb

Re: help me about creating cluster by pelican hpc iam beginer

2009-09-21T01:54:56Z

Sorry, offhand I don't know what the process name is when the atftpd daemon is started. Whatever it is, it is running after you do pelican_setup. I went back and looked at your original message, and I see you're using a hub. Why don't you try to connect the frontend and a single compute node using a crossover cable? If the hub is managed, it can mess things up by handing out IP addresses. An unmanaged switch is what I use.

Re: help me about creating cluster by pelican hpc iam beginer

2009-09-21T01:34:28Z

but if tftp not running how i could to get pxelinux.0 in
/var/lib/tftpboot with tftp client on WindowsXp .

Re: help me about creating cluster by pelican hpc iam beginer

2009-09-21T01:29:19Z

#Hi
this is very very odd to me
i understood that tftp is not run .
i test this :
# /etc/init.d/atftpd start then i do #ps -aux but i don't saw tftp or ftp or atftp
then i act this
#sudo ps -aux but again i dont see tftp proccess
then i change the 'USE_INETD=true' to 'USE_INETD=false' in '/etc/init.d/atftpd'
and then /etc/init.d/atftpd start then i get #ps but again not exist.
at last i test this /usr/sbin/atftpd --daemon /var/lib/tftpboot
and again i get #ps -aux but tftp process not exist .

this odd why tftp process not run?
well when i run commands for example :
#/usr/sbin/atftpd --daemon /var/lib/tftpboot
i see command prompt means ' # ' and this is no error.
what happened ? what i should to do ?
i use live cd linux
best regard
Hogat

Re: Using C on PelicanHPC

2009-09-21T00:47:06Z

OK, the problem with running this from /media/disk is that the executable is available only to the frontend, but not the compute nodes. Doing it from /home/user solves the problem, because that directory is NFS exported.

For your work, I don't see any reason why PelicanHPC shouldn't work well. You will definitely want to mount external storage (during the initial boot up, replace "ram1" with something like "sda1", where that is the device name of a partition formatted as ext2 or ext3. For speeding up the work, I have a couple of ideas. If it is possible to generate all the information needed to plot a frame ahead of time, do that, and then plot complete frames on nodes. If you absolutely need to have the .bmp file (and not just the input parameters) before the information needed to plot the next frame can be computed, then slice the frames into blocks, and compute blocks on nodes. This might be complicated if the pixels on either size of a slice are not independent. That's the reason I would try to do the first approach.

Plotting and video rendering is pretty widespread, but I don't know much about it. You might try searching for some topic-specific mail lists to get advice.

Cheers, M.