Progress Report (2002/04/10) - Kevin Chu

I've been working on the underlying infrastructure to support the distributed computing framework. The main Java components that needed to be leveraged were:

  1. Networking
  2. Threads
  3. Remote Object Invokation (RMI)
  4. Dynamic class loading

Using these components, I've implemented a server daemon that allows a client to submit "jobs" (through a client-side library) and have them run on the server. Right now there is a Job object that resides on the server and handles all the requests from the client (and eventually from the volunteer nodes) regarding the Job. No special networking/threads for this part because that is all taken care of by RMI. Two key features of the Job object are:

  1. It uses a dynamic class loader (I needed to write our own since the java.lang.ClassLoader is abstract and the normal class loader didn't seem to allow dynamically changing the classpath).
  2. It transfers data files and .class files from the client node to the server so that the server can run the job. Eventually, this will also be transferred to the volunteer nodes so that the actual computation takes place on the volunteer nodes.

The final implementation/design of the Job class and what it does to get a job running will depend on the final job and task "specifications" (Kunal is working on this at the moment).

Open Issues

  1. We will have to come up with a special way to access to data files because the file names will probably be changed as the data is moved from the client to the volunteer nodes. This is necessary to avoid name collisions from different jobs. Right now, the reason that we can't just create directories for each job is that the SecurityManager doesn't allow access to those newly created files because they weren't in the original java.policy file that sets up the security policy. I have to look into this some more to see if there is a way to dynamically change the permisssions to access those newly created directories. Another possibility is that we subclass the SecurityManager class and do some custom checking for file access requests. I haven't yet looked into how much work this would be.
  2. Job and Task specifications. I need to know how jobs and tasks are specified so that I can build the daemons to handle these correctly.
  3. Volunteer node registration. I just have to spend some time building this code to do this. I've already started a VolunteerManager class that is intended to manage this aspect of the framework.
  4. Intertask communication. This is is on the wish list (not critical path). I haven't had a chance to really look into this since I last discussed it with Kunal.

Last Comment

Why Grapevine? I guess I like it because we're trying to make little nodes that are hanging off the internet do work. It's kind of like grapes. From their appearance, they aren't doing anything on the vine; they are just idly sitting there in the shade. But really, they are working to store energy from the leaves into sugars and other materials for the next generation of grapevine.