Use Java 20 as bootjdk for jdknext #17302

keithc-ca · 2023-04-27T17:01:57Z

keithc-ca · 2023-04-27T17:02:37Z

In draft mode until we ensure build machines are ready for this change.

pshipton · 2023-04-27T17:44:03Z

I don't think this will work. It downloads the JVM from https://api.adoptopenjdk.net/v3/binary/latest/${bootJDKVersion}/ga/${os}/${arch}/jdk/openj9/normal/adoptopenjdk?project=jdk for which there is no openj9 version of 20 (or even 19) yet.

pshipton · 2023-04-27T17:55:18Z

You can modify the download link. I wrote some ramblings about that in #17289 (comment)

keithc-ca · 2023-04-27T19:26:49Z

This new version updates the URL for downloading (new) boot JDKs.
The old URL pattern doesn't seem to work even for versions for which it could redirect to a Semeru build of OpenJ9.

pshipton · 2023-04-27T20:13:22Z

buildenv/jenkins/variables/defaults.yml

    dir_strip:
      all: '1'
    url:
-      all: "https://api.adoptopenjdk.net/v3/binary/latest/${bootJDKVersion}/ga/${os}/${arch}/jdk/openj9/normal/adoptopenjdk?project=jdk"
+      all: 'https://api.adoptium.net/v3/binary/latest/${bootJDKVersion}/ga/${os}/${arch}/jdk/hotspot/normal/eclipse?project=jdk'


We should only be using hotspot for next and openj9 for the others.

I tried using "openj9" for 17, I got a 404 error.

Perhaps @AdamBrousseau knows, I assume it used to work. It's only needed when we have a new machine that doesn't already have the binary. We could leave versions earlier than next as-is for now in order to deliver the jdk21 fix.

Adam explained that while adoptopenjdk.net redirects to adoptium.net, the redirection doesn't apply to api.adoptopenjdk.net. I'll update this accordingly.

pshipton · 2023-04-27T20:58:06Z

buildenv/jenkins/variables/defaults.yml

@@ -163,11 +163,11 @@ boot_jdk_default:
      17: '17'
      19: '18'
      20: '18'
-      next: '18'
+      next: 'https://api.adoptium.net/v3/binary/latest/20/ga/${os}/${arch}/jdk/hotspot/normal/eclipse?project=jdk'


Does this work? I think you need to put the version here and the url under url:

I think you're right, perhaps this section should look like this?

boot_jdk_default: boot_jdk: location: all: '${HOME}/bootjdks' version: 8: '8' 11: '11' 17: '17' 19: '18' 20: '18' next: '20' dir_strip: all: '1' url: all: 'https://api.adoptopenjdk.net/v3/binary/latest/${bootJDKVersion}/ga/${os}/${arch}/jdk/openj9/normal/adoptopenjdk?project=jdk' next: 'https://api.adoptium.net/v3/binary/latest/${bootJDKVersion}/ga/${os}/${arch}/jdk/hotspot/normal/eclipse?project=jdk'

@AdamBrousseau ?

Looks good to me, I believe we've done something like that before.

pshipton · 2023-04-28T14:33:58Z

jenkins compile amac jdknext

pshipton · 2023-04-28T14:48:02Z

Unfortunately it doesn't work, I guess you'll have to specify all the individual versions rather than setting all.

pshipton · 2023-04-28T14:48:48Z

Either than or perhaps the code that uses it could be fixed to recognize the version before using all.

keithc-ca · 2023-04-28T15:01:55Z

I expect @AdamBrousseau can confirm, but I don't think this change is included in the early stages of the build and so cannot affect the bootjdk URL.

keithc-ca · 2023-04-28T15:04:49Z

Either than or perhaps the code that uses it could be fixed to recognize the version before using all.

I think the version-specific URL would be fine if this change were loaded early enough; see https://github.com/eclipse-openj9/openj9/blob/master/buildenv/jenkins/common/variables-functions.groovy#L1603.

pshipton · 2023-04-28T15:13:34Z

We can see what happened in the console output https://openj9-jenkins.osuosl.org/job/Build_JDKnext_aarch64_mac_Personal/75/console

It does the following before using the variable file.

10:38:04 Checking out Revision d9155d3f38a496aa2cc333e96bb65f3930148a60 (refs/remotes/origin/pr/17302/merge)

Did you look at how getScalarField is implemented?

AdamBrousseau · 2023-04-28T15:19:04Z

buildenv/jenkins/variables/defaults.yml

    dir_strip:
      all: '1'
    url:
-      all: "https://api.adoptopenjdk.net/v3/binary/latest/${bootJDKVersion}/ga/${os}/${arch}/jdk/openj9/normal/adoptopenjdk?project=jdk"
+      all: 'https://api.adoptopenjdk.net/v3/binary/latest/${bootJDKVersion}/ga/${os}/${arch}/jdk/openj9/normal/adoptopenjdk?project=jdk'
+      next: 'https://api.adoptium.net/v3/binary/latest/${bootJDKVersion}/ga/${os}/${arch}/jdk/hotspot/normal/eclipse?project=jdk'


I think next needs to be 20. L1603 is using bootJDKVersion which has already been processed into a number at that point.

Ah, I think I see what you mean; '20': {url} - that makes sense.

Updated in 93021cf.

Fetch bootjdk for jdknext from adoptium. Issue: 17289. Signed-off-by: Keith W. Campbell <[email protected]>

keithc-ca · 2023-04-28T15:29:15Z

I don't think this change is included in the early stages of the build

Clearly I was incorrect, the build tried to use 20 for the boot jdk, but didn't have the right mapping to a URL.

pshipton · 2023-04-28T16:09:45Z

jenkins compile amac jdknext

pshipton · 2023-04-28T16:19:01Z

Seems we need more changes to handle other builds, I see

12:14:49  ++ ls
12:14:49  ++ grep semeru
12:14:49  + sdkFile=

keithc-ca · 2023-04-28T17:02:04Z

jenkins compile amac jdknext

AdamBrousseau · 2023-04-28T17:18:36Z

buildenv/jenkins/common/variables-functions.groovy

@@ -1618,7 +1618,7 @@ def download_boot_jdk(bootJDKVersion, bootJDK) {
        sh """
            curl -LJkO ${sdkUrl}
            mkdir -p ${bootJDK}
-            sdkFile=`ls | grep semeru`
+            sdkFile=`ls | grep 'tar\|zip'`


Might need 2 back slashes?

Using single quotes keeps the shell from giving the backslash special meaning; I guess it has to get past groovy first.

Signed-off-by: Keith W. Campbell <[email protected]>

keithc-ca · 2023-04-28T17:22:46Z

jenkins compile amac jdknext

keithc-ca · 2023-04-28T17:25:04Z

Looks good now:

13:24:13  ++ grep 'tar\|zip'
13:24:13  + sdkFile=OpenJDK20U-jdk_aarch64_mac_hotspot_20.0.1_9.tar.gz
13:24:13  + [[ OpenJDK20U-jdk_aarch64_mac_hotspot_20.0.1_9.tar.gz == *zip ]]
13:24:13  + gzip -cd OpenJDK20U-jdk_aarch64_mac_hotspot_20.0.1_9.tar.gz
13:24:13  + tar xof - -C /Users/jenkins/bootjdks/jdk20 --strip=3
13:24:14  + /Users/jenkins/bootjdks/jdk20/bin/java -version
13:24:15  openjdk version "20.0.1" 2023-04-18
13:24:15  OpenJDK Runtime Environment Temurin-20.0.1+9 (build 20.0.1+9)
13:24:15  OpenJDK 64-Bit Server VM Temurin-20.0.1+9 (build 20.0.1+9, mixed mode)

keithc-ca · 2023-04-28T17:25:26Z

jenkins compile win jdknext

keithc-ca · 2023-04-28T17:27:33Z

The Windows build failed:

13:26:51  + mkdir -p /home/jenkins/bootjdks/jdk20
13:26:51  mkdir: cannot create directory '/home/jenkins/bootjdks/jdk20': Permission denied

@AdamBrousseau Why should that be?

AdamBrousseau · 2023-04-28T17:39:57Z

Looks like that dir is owned by Administrator on the new machines.
Fixed them now.

AdamBrousseau · 2023-04-28T17:40:32Z

I restarted your running win build.

keithc-ca · 2023-04-28T17:44:44Z

Is this another feature of new machines?

13:41:24  + git clone -b openj9 https://github.com/ibmruntimes/openj9-openjdk-jdk.git . --reference /home/jenkins/openjdk_cache
13:41:24  fatal: destination path '.' already exists and is not an empty directory.

keithc-ca · 2023-05-02T13:37:30Z

@AdamBrousseau Has the problem on Windows build machines that caused the failure noted in #17302 (comment) been addressed?

AdamBrousseau · 2023-05-02T13:56:24Z

jenkins compile win jdknext

keithc-ca requested review from pshipton and AdamBrousseau April 27, 2023 17:02

keithc-ca force-pushed the bootjdk branch from 2f8539a to f5a9aab Compare April 27, 2023 19:22

keithc-ca marked this pull request as ready for review April 27, 2023 19:26

pshipton reviewed Apr 27, 2023

View reviewed changes

keithc-ca force-pushed the bootjdk branch from f5a9aab to 2ab58ba Compare April 27, 2023 20:49

pshipton reviewed Apr 27, 2023

View reviewed changes

keithc-ca force-pushed the bootjdk branch from 2ab58ba to 7618413 Compare April 27, 2023 21:14

AdamBrousseau reviewed Apr 28, 2023

View reviewed changes

Use Java 20 as bootjdk for jdknext

93021cf

Fetch bootjdk for jdknext from adoptium. Issue: 17289. Signed-off-by: Keith W. Campbell <[email protected]>

keithc-ca force-pushed the bootjdk branch from 7618413 to 93021cf Compare April 28, 2023 15:27

AdamBrousseau reviewed Apr 28, 2023

View reviewed changes

Support non-Semeru boot JDKs

eaa7e5d

Signed-off-by: Keith W. Campbell <[email protected]>

keithc-ca force-pushed the bootjdk branch from 57e7f35 to eaa7e5d Compare April 28, 2023 17:22

pshipton approved these changes Apr 28, 2023

View reviewed changes

AdamBrousseau approved these changes May 2, 2023

View reviewed changes

AdamBrousseau merged commit a26b88f into eclipse-openj9:master May 2, 2023

keithc-ca deleted the bootjdk branch May 2, 2023 17:19

keithc-ca mentioned this pull request May 3, 2023

Builds failing in openj9-staging branch of jdk21 #17289

Closed

Use Java 20 as bootjdk for jdknext #17302

Use Java 20 as bootjdk for jdknext #17302

Conversation

keithc-ca commented Apr 27, 2023

keithc-ca commented Apr 27, 2023

pshipton commented Apr 27, 2023

pshipton commented Apr 27, 2023

keithc-ca commented Apr 27, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pshipton commented Apr 28, 2023

pshipton commented Apr 28, 2023

pshipton commented Apr 28, 2023

keithc-ca commented Apr 28, 2023

keithc-ca commented Apr 28, 2023

pshipton commented Apr 28, 2023

AdamBrousseau Apr 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

keithc-ca commented Apr 28, 2023

pshipton commented Apr 28, 2023

pshipton commented Apr 28, 2023

keithc-ca commented Apr 28, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

keithc-ca commented Apr 28, 2023

keithc-ca commented Apr 28, 2023

keithc-ca commented Apr 28, 2023

keithc-ca commented Apr 28, 2023

AdamBrousseau commented Apr 28, 2023

AdamBrousseau commented Apr 28, 2023

keithc-ca commented Apr 28, 2023

keithc-ca commented May 2, 2023

AdamBrousseau commented May 2, 2023

AdamBrousseau Apr 28, 2023 •

edited

Loading