Progress Report (2002/04/10) - Kevin Chu
I've been working on the underlying infrastructure to support the distributed
computing framework. The main Java components that needed to be
leveraged were:
- Networking
- Threads
- Remote Object Invokation (RMI)
- Dynamic class loading
Using these components, I've implemented a server daemon that allows
a client to submit "jobs" (through a client-side library) and have them
run on the server. Right now there is a Job object that resides on the
server and handles all the requests from the client (and eventually
from the volunteer nodes) regarding the Job. No special networking/threads
for this part because that is all taken care of by RMI. Two key features
of the Job object are:
- It uses a dynamic class loader (I needed to write our own since the
java.lang.ClassLoader is abstract and the normal class loader didn't seem
to allow dynamically changing the classpath).
- It transfers data files and .class files from the client node to
the server so that the server can run the job. Eventually, this
will also be transferred to the volunteer nodes so that the actual
computation takes place on the volunteer nodes.
The final implementation/design of the Job class and what it does to
get a job running will depend on the final job and task "specifications"
(Kunal is working on this at the moment).
Open Issues
- We will have to come up with a special way to access to data files
because the file names will probably be changed as the data is moved from
the client to the volunteer nodes. This is necessary to avoid name
collisions from different jobs. Right now, the reason that we can't
just create directories for each job is that the SecurityManager doesn't
allow access to those newly created files because they weren't in the
original java.policy file that sets up the security policy. I have to
look into this some more to see if there is a way to dynamically change
the permisssions to access those newly created directories. Another
possibility is that we subclass the SecurityManager class and do some
custom checking for file access requests. I haven't yet looked into how
much work this would be.
- Job and Task specifications. I need to know how jobs and tasks
are specified so that I can build the daemons to handle these correctly.
- Volunteer node registration. I just have to spend some time building
this code to do this. I've already started a VolunteerManager class that
is intended to manage this aspect of the framework.
- Intertask communication. This is is on the wish list (not critical
path). I haven't had a chance to really look into this since I last
discussed it with Kunal.
Last Comment
Why Grapevine? I guess I like it because we're trying to make little
nodes that are hanging off the internet do work. It's kind of like
grapes. From their appearance, they aren't doing anything on the vine;
they are just idly sitting there in the shade. But really, they are
working to store energy from the leaves into sugars and other materials
for the next generation of grapevine.