Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

Expected behavior (Ubuntu 10.04 LTS):

mike@boontoo10:~$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

mike@boontoo10:~$ java -Xmx512m -Xms512m -jar forker.jar 10000000 20 Xmx512m Xms512m

output of "top":

1928 mike      20   0  694m  54m 8624 S    6  5.4   0:00.18 java
1933 mike      20   0  694m  54m 8648 S    6  5.4   0:00.18 java
1935 mike      20   0  694m  54m 8648 S    6  5.4   0:00.18 java
1939 mike      20   0  694m  54m 8648 S    5  5.4   0:00.16 java

(the java7 output looks like this on 10.04):

mike@ubuntu10java7:~$ java -versionjava version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
 2106 mike      20   0  734m  57m 9876 S    4  2.8   0:00.25 java               
 2091 mike      20   0  734m  57m 9876 S    3  2.8   0:00.26 java               
 2100 mike      20   0  734m  57m 9876 S    3  2.8   0:00.25 java               
 2112 mike      20   0  734m  57m 9876 S    3  2.8   0:00.24 java               
 2097 mike      20   0  734m  57m 9876 S    3  2.8   0:00.24 java               

New behavior (Ubuntu 12.04 LTS):

mike@boontoo12:~$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

mike@boontoo12:~$ java -Xmx512m -Xms512m -jar forker.jar 10000000 20 Xmx512m Xms512m

output of "top":

20697 mike      20   0 1405m  54m 8488 S    5  1.4   0:00.18 java
20669 mike      20   0 1405m  54m 8492 S    5  1.4   0:00.18 java
20691 mike      20   0 1405m  54m 8492 S    5  1.4   0:00.17 java
20658 mike      20   0 1405m  54m 8492 S    5  1.4   0:00.17 java

Update: New behavior (Ubuntu 12.04 LTS w/ Java 7):

mike@boontoo12java7:~$ java -version
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)

output of "top":

 4166 mike      20   0 1438m  56m 9572 S    7  2.8   0:00.20 java               
 4169 mike      20   0 1438m  56m 9572 S    7  2.8   0:00.20 java               
 4182 mike      20   0 1438m  56m 9572 S    7  2.8   0:00.20 java               
 4076 mike      20   0 1438m  56m 9572 S    6  2.8   0:00.19 java               
 4100 mike      20   0 1438m  56m 9572 S    6  2.8   0:00.19 java           

ctd... 12.0.4 with OpenJDK 6, similar problem:

mike@boontoo12:~$ java -version
java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.6) (6b27-1.12.6-1ubuntu0.12.04.2)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

output of "top":

 6690 mike      20   0 1500m  55m 8920 S    7  1.4   0:00.22 java               
 6692 mike      20   0 1500m  55m 8920 S    7  1.4   0:00.21 java               
 6697 mike      20   0 1500m  55m 8920 S    7  1.4   0:00.21 java               
 6701 mike      20   0 1500m  51m 8684 S    7  1.3   0:00.21 java               
 6725 mike      20   0 1500m  55m 8920 S    7  1.4   0:00.21 java               

To preempt, here's the OpenJDK 7 attempt:

mike@boontoo12:~$ java -version
java version "1.7.0_25"
OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1ubuntu0.12.04.2)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)

output of "top":

 9327 mike      20   0 1534m  57m 9296 S    6  1.4   0:00.25 java               
 9331 mike      20   0 1534m  57m 9276 S    6  1.4   0:00.25 java               
 9344 mike      20   0 1534m  57m 9300 S    6  1.4   0:00.25 java               
 9347 mike      20   0 1534m  57m 9300 S    6  1.4   0:00.25 java               
 9323 mike      20   0 1534m  57m 9300 S    6  1.4   0:00.25 java               
 9284 mike      20   0 1534m  57m 9308 S    5  1.4   0:00.24 java               
 9285 mike      20   0 1534m  57m 9300 S    5  1.4   0:00.24 java               
 9336 mike      20   0 1534m  57m 9276 S    5  1.4   0:00.23 java               
 9277 mike      20   0 1534m  57m 9300 S    5  1.4   0:00.24 java          

As you can see, the vmem utilization has doubled. I'm using the exact same java/jre binary (and jdk), the only difference being the version of Ubuntu.

I'm hoping to get this program back to ~694m of vmem used. How do I do that? What do I even search for?

Please don't suggest that I structure my program differently. This is the smallest toy-example I could write to reproduce a gigantic problem reflected in some Apache Software, but this feels like a kernel/libs problem to me.

Here's the code to duplicate, and the link where all this is summarized in a gist: https://gist.github.com/michael-trelinski

Code:

package com.satoricode.forker;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class ForkIt {


  public static void main(String[] args) throws InterruptedException {
        int arraySize = Integer.parseInt(args[0]);
        System.out.print("Filling up some memory... ");
        int[] arr = fillItUp(arraySize);
        System.out.println("" + arr.length + " done.");
        if (args.length == 1) {
            System.out.println("I'm the fork!");
            Thread.sleep(10 * 1000);
        } else {
            // I'm the master
            int numForks = Integer.parseInt(args[1]);
            String stackCommand = args[2];
            String stackInitCommand = args[3];
            ExecutorService pool = Executors.newFixedThreadPool(numForks);

            System.out.println("Launching " + numForks + " forks with array size " + arraySize + " with -" + stackCommand + " -" + stackInitCommand);
            for(int I = 0; I < numForks; I++)
                pool.execute(new ForkRunner(arraySize, stackCommand, stackInitCommand));

            pool.shutdown();
            System.out.print("Waiting for pool to finish");
            while(!pool.isTerminated()) {
                Thread.sleep(50);
                System.out.print(".");              
            }
            System.out.println();
            System.out.println("done\n");           
        }
    }

    private static int[] fillItUp(int total) {
        int[] arr = new int[total];
        for (int I = 0; I < total; I++)
            arr[I] = I;
        return arr;
    }

}


package com.satoricode.forker;

import java.io.IOException;
import java.io.InputStream;


public class ForkRunner implements Runnable {

  private String stackCommand;
    private int arraySize;
    private String stackInitCommand;

    public ForkRunner(int arraySize, String stackCommand, String stackInitCommand) {
        this.arraySize = arraySize;
        this.stackCommand = stackCommand;
        this.stackInitCommand = stackInitCommand;
    }

    @Override
    public void run() {
        try {
            String command = "java -" + stackCommand + " -" + stackInitCommand + " -jar forker.jar " + arraySize;
            System.out.println("executing: " + command);
            Process proc = Runtime.getRuntime().exec(command);
            InputStream input = proc.getInputStream();
            InputStream error = proc.getErrorStream();
            StringBuilder sb = new StringBuilder();
            int c = -1;
            while ((c = input.read()) != -1) {
                sb.append((char) c);
            }
            while ((c = error.read()) != -1) {
                sb.append((char) c);
            }
            System.out.println(sb.toString());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}
share|improve this question

closed as off-topic by Luis Jul 26 at 17:14

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This describes a problem that can't be reproduced that seemingly went away on its own or was only relevant to a very specific period of time. It's off-topic as it's unlikely to help future readers." – Luis
If this question can be reworded to fit the rules in the help center, please edit the question.

    
mm... Do you get the same results in Java7? JDK6 or 7? –  Braiam Sep 19 '13 at 17:29
    
@Braiam I will try that out, but I suspect since I used the same java binary to install on both Ubuntu versions, that it's something os/kernel-based. –  Mike Trelinski Sep 19 '13 at 17:32
    
IMHO, you shouldn't be using Java6 on recent versions, is quite buggy and unmaintained so maybe is just the combination recent kernel + old java. –  Braiam Sep 19 '13 at 17:34
    
@Braiam I just did a vanilla 12.04 server install and did a sun java jre 1.7 install. Nearly identical problem. Updated original post. Will try out java 7 on 10.04 and see if I get a similar result from java 6 on 10.4... –  Mike Trelinski Sep 19 '13 at 18:10
1  
OpenJDK isn't (very) condoned for running Apache's Hadoop, but like I keep saying, I really believe this is a problem with fork() at the kernel/source-level. I will test this out with OpenJDK 6 and 7, and post the results later. I anticipate that they will be the same. –  Mike Trelinski Sep 20 '13 at 0:15

Browse other questions tagged or ask your own question.