Monday, 2 February 2015

progress bar for file upload with Apache HttpClient 4 and StringEntity

There is a similar solution on stackoverflow, and it uses MultipartEntity to upload a file. However, if you are required to use StringEntity and put file data into JSON POST request like what I needed to do, please see my source code below.



code extract to use HttpPost with StringEntity with JSON in it.
        String jsonRequest = "the file data and related stuff";  
        StringEntity stringEntityForProgressBar = new StringEntity(jsonRequest, progressBean);  
        httpPost.setEntity(stringEntityForProgressBar);  
        HttpResponse response = httpClient.execute(httpPost);  



My own StringEntity.java

 import java.io.IOException;  
 import java.io.OutputStream;  
 import java.io.UnsupportedEncodingException;  
 import com.edentiti.registrations.verification.ProgressBean;  

 public class StringEntity extends org.apache.http.entity.StringEntity  
 {  
      private ProgressBean _progressBean;  
      public StringEntity(String string, ProgressBean progressBean) throws UnsupportedEncodingException  
      {  
           super(string);  
           _progressBean = progressBean;  
      }  
      private OutputStreamProgress outstream;  
      @Override  
      public void writeTo(OutputStream outstream) throws IOException  
      {  
           this.outstream = new OutputStreamProgress(outstream, _progressBean, getContentLength());  
           super.writeTo(this.outstream);  
      }  
 }  



My own StringEntity.java, for my own purpose, the progress goes up to 70% here and you can adjust it(e.g. up to 100%) to suit your own purpose.

 import java.io.IOException;  
 import java.io.OutputStream;  
 import com.edentiti.registrations.verification.ProgressBean;  

 public class OutputStreamProgress extends OutputStream {  
      private final static int CHUNK_SIZE = 40960; // Ming feels it is a lucky number, no solid reason for it.  
      private ProgressBean _progressBean;  
      private long _contentLength;  
   private final OutputStream outstream;  
   private volatile long bytesWritten=0;  
      public OutputStreamProgress(OutputStream outstream, ProgressBean progressBean, long contentLength)  
      {  
           this._progressBean = progressBean;  
           this._contentLength = contentLength;  
     this.outstream = outstream;  
   }  
   @Override  
   public void write(int b) throws IOException {  
     outstream.write(b);  
     bytesWritten++;  
   }  
   @Override  
   public void write(byte[] b) throws IOException {  
           int numChunks;  
           if (b.length % CHUNK_SIZE == 0)  
           {  
                numChunks = b.length / CHUNK_SIZE;  
           }  
           else  
           {  
                numChunks = (b.length / CHUNK_SIZE) + 1;  
           }  
           for (int i = 0; i <= numChunks; i++)  
           {  
                // if last block  
                if (i == numChunks)  
                {  
                     write(b, i * CHUNK_SIZE, b.length);  
                }  
                else  
                {  
                     write(b, i * CHUNK_SIZE, CHUNK_SIZE);  
                }  
           }  
   }  
   @Override  
   public void write(byte[] b, int off, int len) throws IOException {  
     outstream.write(b, off, len);  
     bytesWritten += len;  
           long progressValue = (long) ((70 * getWrittenLength()) / _contentLength);  
           if (progressValue > 70)  
           {  
                progressValue = 70;  
           }  
           if (progressValue < 1)  
           {  
                progressValue = 1;  
           }  
           // here we only allow something between 1 to 70  
           _progressBean.setValue(Long.valueOf(progressValue));  
   }  
   @Override  
   public void flush() throws IOException {  
     outstream.flush();  
   }  
   @Override  
   public void close() throws IOException {  
     outstream.close();  
   }  
   public long getWrittenLength() {  
     return bytesWritten;  
   }  
 }  


References:

  1. http://stackoverflow.com/questions/7057342/how-to-get-a-progress-bar-for-a-file-upload-with-apache-httpclient-4

Sunday, 1 February 2015

seam richfaces progress bar

What I want to achieve here is:


After the end user clicks on 'confirm and submit' button, the server processes the input which could take minutes. While the end user is waiting, we show a richfaces modal with a progress bar to indicate the progress. When the processing is done, server redirects end user to the processing result URL.

 <h:form id="form-passportupload">  
           <a4j:commandButton value="confirm and submit" action="#{progressBarVerifier.updateDocumentAsync()}" oncomplete="#{progressBarVerifier.isPageValidated}?Richfaces.showModalPanel('progressBarModal'):Richfaces.hideModalPanel('progressBarModal');"  
                          ignoreDupResponses="true" ajaxSingle="true" reRender="progressBarPanel,fileupload-error,fileupload-terms-confirm">  
           </a4j:commandButton>  
 </h:form>  


      <rich:modalPanel id="progressBarModal">  
           <h:form>  
                <a4j:outputPanel id="progressBarPanel">  
                Please wait while we check your uploaded document. This may take up to 5 minutes.  
                <rich:progressBar value="#{progressBarVerifier.progressBean.value}" label="#{progressBarVerifier.progressBean.value} %" minValue="0" maxValue="100" timeout="300000" interval="3000" enabled="#{progressBarVerifier.progressBean.enabled}" >  
                  <f:facet name="initial">  
                    <h:outputText value="Processing" />  
                  </f:facet>  
                  <f:facet name="complete">  
                    <h:outputText value="Process completed">  
                         #{redirectHelper.redirectToVerifySinglePage()}  
                    </h:outputText>  
                  </f:facet>  
                </rich:progressBar>  
                </a4j:outputPanel>  
           </h:form>  
      </rich:modalPanel>  



ProgressBean looks like this. Please note the value is updated in updateDocumentAsync() method. At the back end, we use Httpclient to do the POST and update ProgressBean.value in the process.


 public class ProgressBean  
 {  
      private boolean _enabled = false;  
      public boolean isEnabled()  
      {  
           return _enabled;  
      }  
      public void setEnabled(boolean enabled)  
      {  
           _enabled = enabled;  
      }  
      private Long value = new Long(0);  
      public Long getValue()  
      {  
           return value;  
      }  
      public void setValue(final Long value)  
      {  
           this.value = value;  
      }  
 }  


asyncUpload() method has to be Seam @Asynchronous and you can do server end validation before it. Otherwise, progress bar will not start to update until updateDocumentAsync() has finished.



 public void updateDocumentAsync() throws Exception  
 {  
   _validator.doValidation();  
   _asyncUploader.asyncUpload();  
 }  




Please see attached screen shot below for the results.





References:
  1. https://achorniy.wordpress.com/2010/10/22/show-dynamic-process-progress-in-seam-richfaces/ 
  2. http://forkbomb-blog.de/2011/time-consuming-processes-with-seam-richfaces-a4jpoll-and-quartz 
  3. http://docs.jboss.org/richfaces/latest_3_3_X/en/devguide/html/rich_progressBar.html

Saturday, 13 December 2014

FTP download file using Bash script

Recently, I need to download a file daily from external FTP site, process the downloaded file locally and store the result into database. Below is the part of bash script to download a file. Just run the Bash file with Cron, job well done.




cd $DIR_TO_STORE_FILE_DOWNLOADED

# Call 1. Uses the ftp command with the -inv switches.  -i turns off interactive prompting. -n Restrains FTP from attempting the auto-login feature. -v enables verbose and progress.

ftp -inv $HOST << EOF

# Call 2. Here the login credentials are supplied by calling the variables.
user $USER $PASS
get $FILE_NAME_TO_DOWNLOAD
bye
EOF

Sunday, 9 November 2014

Java exception: null value was assigned to a property of primitive type setter



When adding a new property to an existing entity, Java might complain that 'null value was assigned to a property of primitive type setter' at run time.
The reason is the existing entities have null value in the database for the newly added column.


The SQL query to add a property might look like below:

alter table registration add column cost double precision;

One way to overcome this is to use below get method instead of the standard one.




 public void setCost(final Double cost)  
 {  
  _cost = cost;  
 }  
 public static Double getCost(final Double cost)  
 {  
  if (cost == null)  
  {  
   return new Double(0.0);  
  }  
  else  
  {  
   return cost;  
  }  
 }  




Another way would be setting default value for the newly field in the database, which might not always easy for a large live production database. The SQL query might look like below:


alter table registration add column cost double precision default 0

Friday, 31 October 2014

plain html file which parses request URL params and use them in post form

Right before Melbourne Cup 2014, there is a requirement to redirect end users of our customers to external URL.

The end user would see 'intro.html'. Within this page, the user can click on a button in a form. Clicking this button would redirect the end user to an external URL.

The external URL would be passed in URL request, something like
intro.html?userId=123456&cancelUrl=http://www.google.com

The form needs to extract 'userId' and 'cancelUrl' from the URL.





As for the front end html file, the only tricky bit is to use javascript to extract some parameters from URL


 <form id="cancelurl-form" name="retry-form" method="get" onsubmit="get_action(this);">  
   <input id="userid-input" type="hidden" name="userId" />  
   <input class="submit" value="Skip identification" type="submit" />  
 </form>  
 <script type="text/javascript">  
 function getUrlParameter(sParam)  
 {  
   var sPageURL = window.location.search.substring(1);  
   var sURLVariables = sPageURL.split('&');  
   for (var i = 0; i < sURLVariables.length; i++)  
   {  
     var sParameterName = sURLVariables[i].split('=');  
     if (sParameterName[0] == sParam)  
     {  
       return sParameterName[1];  
     }  
   }  
 }  
 function get_action(form) {  
  if (!getUrlParameter('cancelUrl'))  
  {  
  form.action = '/verification/prefuse/intro.html';   
  }  
  else  
  {  
  form.action = getUrlParameter('cancelUrl');  
  }  
 }  
 </script>  
 <script type="text/javascript">  
  var elem = document.getElementById("userid-input");  
  elem.value = getUrlParameter('userId');  
 </script>  



We normally use org.jboss.seam.faces.Redirect object to redirect to our JSF xhtml page.
Since the destination URL is a pure html, the back end JAVA code need to be like below.



 FacesContext context = FacesContext.getCurrentInstance();  
 ExternalContext externalContext = context.getExternalContext();  
 HttpServletResponse response = (HttpServletResponse)externalContext.getResponse();  
 response.sendRedirect("/verification/prefuse/intro.html");  
 context.responseComplete();