Today I was trying to create an application that reads data from a website. The problem I had was that the form I needed to fill in to get to the page I want to read is a page with a POST form.
Due to the fact it was a POST form I could not create a simple URLConnection with a url and the querystring parameters that were needed.
To solve this problem you need to create a URLConnection and write the additional POST parameters to the connection.
First of all the basic code of how to create a URLConnection and read the file line by line.
try { // Send data URL url = new URL("http://cedricascoop.be/Search.aspx?__ufps=066834&l=NL&s=1&p=1"); URLConnection conn = url.openConnection(); // Get the response BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream())); String line; while ((line = rd.readLine()) != null) { // Process line... System.out.println(line); } rd.close(); } catch (Exception e) { }
Now if we want to write some parameters we need to create a OutputStreamWriter. Here we can pass on all the post parameters needed. The parameters need to be UTF-8 encoded.
Following some code to create a URLConnection, pass on some parameters and read it out.
try { // Construct data String data = URLEncoder.encode("trainNumberTextBox", "UTF-8") + "=" + URLEncoder.encode("4518", "UTF-8"); data += "&" + URLEncoder.encode("__EVENTTARGET", "UTF-8") + "=" + URLEncoder.encode("searchDepartureCommand", "UTF-8"); data += "&" + URLEncoder.encode("__EVENTARGUMENT", "UTF-8") + "=" + URLEncoder.encode("searchForm", "UTF-8"); data += "&" + URLEncoder.encode("__ET", "UTF-8") + "=" + URLEncoder.encode("", "UTF-8"); // Send data URL url = new URL("http://cedricascoop.bnet.be/Search.aspx?__ufps=066834&l=NL&s=1&p=1"); URLConnection conn = url.openConnection(); conn.setDoOutput(true); OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream()); wr.write(data); wr.flush(); // Get the response BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream())); String line; while ((line = rd.readLine()) != null) { // Process line... System.out.println(line); } wr.close(); rd.close(); } catch (Exception e) { }