Scala Source.fromInputStream not generating full results

PPPP Source

We are working on writing an existing python utility into Scala. Utility downloads compressed data from a REST API as ".gzip" file.

Python code :

        with open(file_path, 'wb+') as f:
            shutil.copyfileobj(response.raw, f) 

While in scala I am writing it this way :

var out: GZIPOutputStream = new GZIPOutputStream(new FileOutputStream(outputFile))
    var writer= new PrintWriter(out)
    try {
      val inputStrem: InputStream = response.getEntity.getContent
      val gzipInputStream = new GZIPInputStream(inputStrem)
      for(line <- Source.fromInputStream(gzipInputStream).getLines){
    } catch {
      case e: Exception => throw e
     }finally {
      if (writer != null) writer.close

But the output file is getting produced of different sizes every time with scala code. Python code is running fine. REST POST and GET requests are same for both utilities. I am not getting any errors in scala , its getting executed successfully but creating different sizes files. Any help would be really appreciated .

Thank You !!



answered 5 days ago Tim #1

If this is Apache HTTP then HttpEntity has a writeTo method that writes the content directly to an OutputStream.

response.getEntity().writeTo(new FileOutputStream(outputFile))

This does not re-encode the data like the original Scala does, but the Python version does not do that either.

comments powered by Disqus