Archive for the ‘Fedora 7’ Category

Condor package for Oscar!

From condor project

The goal of the Condor® Project is to develop, implement, deploy, and evaluate mechanisms and policies that support High Throughput Computing (HTC) on large collections of distributively owned computing resources.
Guided by both the technological and sociological challenges of such a
computing environment, the Condor Team has been building software tools that enable scientists and engineers to increase their computing throughput.

Condor package  for Oscar is currently built around latest stable version 6.8.5. Condor installation is a straightforward way one can use .tar.gz or rpm and new install method is a perl script which has to be provided with many arguments and which configure Condor and make changes to condor_config file they way you want to see condor functioning.

1. I have tested Condor 6.8.5 with FC-5 and FC-6 so need to extract the correct rpm and pack it with package.

There are directly two ways of getting it configured .
Common : Head node as submit and manager & all client nodes in execute mode.

A. Head node in also in execute mode.
B. Head node able to send jobs to other manager. (Not more than one manager for now).
C. Head node able to receive jobs from other managers.

All the three ways above can be easily mixed, but common will always be there by default.

2. I am already done with basic scripts.

what remains for condor package to be released is
a. Convertion to new format (yet to understand how to do that)
b. Uninstaller(which should work in oscar_wizard manage mode, i.e for existing clusters)
c. Testing with other distros
d. test cases (currently performing)

By 18/July Condor package should be available for install with Oscar with all the above configuration modes. There are many more advance configuration of Condor on a cluster I am checking on it and if possible add it to this release but unless very important I won’t like to get them included.
A final release (  that might be first public release) will have to wait some more time which will include more distributions supported by Oscar than just Fedora code 5 & 6, and more testing done on the package.

There will be a issue if we not try to use VDT and configure WS-GRAM for condor scheduling since VDT itself installs Condor as a part luckily there is how to on how to avoid that.

So I am pretty much close with this. I shall make a post when this is over I shall again take over vdt-globus package there is some more learning I did with vdt so will jump directly to configuration since setuping up is almost complete with one issue left which can change the way VDT will install, more on that later.

Technorati Tags: , , , ,

Powered by ScribeFire.

Advertisements

Vdt based globus package for oscar!

I have my Internship with Novell going on and will finish on 15th June so like always there is a semester end rush, something that I am *not* looking forward to encounter in GSoC project though, I have a final presentation and Report before 15th, I will be 100% on the GSoC work after that.

Updates so far..

I have analyzed some aspects for creating globus package for OSCAR, there are still some parts that need to be done so as to get it is done.

I was not frequently blogging but after 15th I shall be more active on providing updates frequently and documenting stuff in OSCAR Wiki.

My GSoC project involve two packages

1. Globus package.
2. Condor package.
( actually third one “gx-map” , but there is already some support for this in VDT more on it in other post)

I would talk about Globus package in this post.

Globus package must have following(I will provided full details soon.)

1. Installation
2. Configuration
3. Uninstallation
4. Support for all distro supported by OSCAR.

There were basically two ways  I was exploring (not explored other available )

1. Using a VDT backed installation. (vdt requires internet connectivity while installation,until there is something like this available)
2. Using created/available  globus rpm’s

We are using VDT which uses pacman for installations and configuration of all its components. We have to be flexible here to make it available all of the following features.

1. VDT’s current cache at (for 1.6.1) http://vdt.cs.wisc.edu/vdt_161_cache , requires internet connectivity.
2. Use locally downloaded vdt snapshot for particular distro.

Q. How to provide local cache for a distro so that machines without net connectivity can deploy globus to their cluster?
Ans.This is actually not very important since anyone planning to install globus will surely have net connectivity, unless their is some reason like research projects. So I will try to do this some time later  after globus package is available for deployment.  So far my idea is to have a local mirror of VDT current version(1.6.1 costs around 2.8 GB) than figure out some script that can make a distro based tar file for download, I am sure it will be far less than the mirror itself 🙂  So that OSCAR users can refer to the local package source rather than remote. more details on how do i plan it here.

I am currently using full mirror so with proper -pretend-platform flag it is working for me during testing on Fedora Core 5  therefore I am not stressing much on creating platform specific pacman VDT cache at this stage.

1. Installation sources : Because of above mentioned reasons I  have to ask user about installation sources for VDT whether they have downloaded it (so that I can ask them about where they did it)  or go to use online VDT.

2. Answering VDT questions : VDT asks several Questions and obviously they can’t be answered at regular prompt so we will try to preanswer them according to package that has been chosen  I will create set of common questions and package specific question so that user need to answer those which are relevant to their installation after dry running vdt based specific installation commands, this is in order to not to confuse user with so many question which might not be relevant to his/her setup.

3. Selecting what to Install : There are many packages which can provide different functionality accordingly like VDT-Gatekeeper, Globus-Server, Globus-WS-Server  there are some overlaps, I am testing vdt command same like in “2” point above to make sure what I am installing.

4. Uninstaller : ./oscarwizard provide facility to removal of packages pacman files can be removed easily but to remove vdt packages installed there might be some additional steps not rpm base dso there is a need to create a uninstaller which will use pacman -remove before removing pacman installation from the Oscar.

5. Configuration of vdt installation : <* now here come real hard part* > configuration is somewhat tedious specially because we need to know what packages are selected for install and which configuration apply.

/**/
configurator.html will be ready by this weekend for sure, 😦 , I have already delayed this much. but this will have all questions to be asked/ installation sources/ and what specifically to install.

/**/
vdt-install.sh will be invoked with proper options set so as to go ahead with installation.  I hope to invoke it with different parameters.

/**/
here only after calling install script I wil run additional set of commands to do proper configuration.

/**/ I know how to go about uninstallaling vdt package but I just need to make sure that
 “pacman -remove”  is called before pacman rpm uninstall, I couldn’t find any post_server_*uninstall script in trunk so have to test this.

hmm that was not so difficult Only stressing is configuration ( that involve lot of ways that grid can be configured so making me think more and more on how users may actually want to configure their resources)

This should complete Globus’s OSCAR package which does not involve any touching of clients since that’s all we will need, (not true with Condor package )

In addition to 5 points there is actually one more.

6. Additional interfaces :to globus for SGE/Condor/PBS   more on this when I will complete the above mentioned task.

and one more

7. Support for multiple distribution.( I am currently downloading CentOS 5, Fedora 7 ) to help in this.

Technorati Tags: , , , , ,

Powered by ScribeFire.