Situation:
We have a small application where you can choose something like "I want Version X.X.X, Nightly, Branch: XYZ" and the Application will fetch the latest package according to your specifications from the build server.
Problem:
The download of the package works in 95 out of 100 cases, but in 5% of all cases the download just stops after a random amount of data is downloaded. When you retry the same thing, it might work or sometimes it might stop at a different amount downloaded. As you might have guessed, this is not a very acceptable rate of success.
The powershell script randomly stops at $response_stream.Read($buffer, 0, $buffer.length) (see script below) after a random amount of data was read. The download speed does not get smaller and smaller and then stops. It just makes a full stop as if the server is not sending anything anymore, and after a few minutes we get a timout error ("Exception when calling "Read" with 3 arguments: The timeout for the process has been exceeded").
I have tried a lot of things, but I'm out of ideas (basically tried to up the timeout on the IIS, tried different approaches wih the powershell download, googled the problem, but all without success). This also happens at random times, at night (very few users) but also during the day (lots of users).
Does anybody have an idea how or what could be tried to make the download more robust? My guess is that somethings wrong with the PHP Script/IIS, but sadly my knowledge with both ist very limited...
Additional Information:
The download on the client side is done via Powershell. I wrote a neat little function which takes care of that:
function Get-FileFromURL
{
param(
[Parameter(Mandatory, Position = 0)] [System.Uri]$URL,
[Parameter(Mandatory, Position = 1)] [string]$OutputFile,
[Parameter(Mandatory, Position = 2)] [System.Management.Automation.PsCredential]$Credentials,
[Parameter(Position = 3)] [string]$Filename
)
try
{
$request = [System.Net.HttpWebRequest]::Create($URL)
$request.Credentials = $Credentials;
$request.set_Timeout(5000) # 5 second timeout
$response = $request.GetResponse()
$total_bytes = $response.ContentLength
$response_stream = $response.GetResponseStream()
$header = $response.Headers.Get("content-disposition");
if($Filename -eq "")
{
$Filename = ($header -split "=")[1].Trim("`"");
$OutputFile = $OutputFile + "" + $Filename;
}
try
{
# 256KB works better on my machine for 1GB and 10GB files
# See https://www.microsoft.com/en-us/research/wp-content/uploads/2004/12/tr-2004-136.pdf
# Cf. https://stackoverflow.com/a/3034155/10504393
$buffer = New-Object -TypeName byte[] -ArgumentList 256KB
$target_stream = [System.IO.File]::Create($OutputFile)
$stopwatch = [system.diagnostics.stopwatch]::StartNew();
$refreshTimerInMilliseconds = 1000;
$refreshTimer = $refreshTimerInMilliseconds / 1000;
do
{
$count = $response_stream.Read($buffer, 0, $buffer.length)
$target_stream.Write($buffer, 0, $count)
$downloaded_bytes = $downloaded_bytes + $count
if ($stopwatch.ElapsedMilliseconds -ge $refreshTimerInMilliseconds)
{
$percent = $downloaded_bytes / $total_bytes;
$speed = ($downloaded_bytes - $prev_downloaded_bytes) / 1KB / $refreshTimer;
$status = @{
completed = "{0,6:p2} Completed" -f $percent
downloaded = "{0:n0} MB of {1:n0} MB" -f ($downloaded_bytes / 1MB), ($total_bytes / 1MB)
speed = "{0,7:n0} KB/s" -f ($speed)
eta = "Time Remaining: {0:hh:mm:ss}" -f (New-TimeSpan -Seconds (($total_bytes - $downloaded_bytes) / 1KB / $speed))
}
$progress_args = @{
Activity = "Downloading '$Filename'"
Status = "$($status.completed) ($($status.downloaded)), $($status.speed), $($status.eta)"
PercentComplete = $percent * 100
}
Write-Progress @progress_args
$prev_downloaded_bytes = $downloaded_bytes
$stopwatch.Restart();
}
} while ($count -gt 0)
$stopwatch.Stop();
}
finally
{
if ($target_stream) { $target_stream.Dispose() }
# If file exists and $count is not zero or $null, then script was interrupted by user
if ((Test-Path $Filename) -and $count) { Remove-Item -Path $Filename }
}
}
finally
{
if ($response) { $response.Dispose() }
if ($response_stream) { $response_stream.Dispose() }
}
}
On the server side we have an IIS Express with a PHP Script that is feeding the file.
if (is_file($file_path))
{
$file_size = filesize($file_path);
$file = @fopen($file_path,"rb");
if ($file)
{
// set the headers, prevent caching
header("Pragma: public");
header("Expires: -1");
header("Cache-Control: public, must-revalidate, post-check=0, pre-check=0");
header("Content-Disposition: attachment; filename="$file_name"");
// set appropriate headers for attachment or streamed file
$is_attachment = isset($_REQUEST['stream']) ? false : true;
if ($is_attachment)
header("Content-Disposition: attachment; filename="$file_name"");
else
header('Content-Disposition: inline;');
// set the mime type based on extension, add yours if needed.
$ctype_default = "application/octet-stream";
$content_types = array(
"exe" => "application/octet-stream",
"zip" => "application/zip",
"mp3" => "audio/mpeg",
"mpg" => "video/mpeg",
"avi" => "video/x-msvideo",
);
$ctype = isset($content_types[$file_ext]) ? $content_types[$file_ext] : $ctype_default;
header("Content-Type: " . $ctype);
//check if http_range is sent by browser (or download manager)
if(isset($_SERVER['HTTP_RANGE']))
{
list($size_unit, $range_orig) = explode('=', $_SERVER['HTTP_RANGE'], 2);
if ($size_unit == 'bytes')
{
//multiple ranges could be specified at the same time, but for simplicity only serve the first range
//http://tools.ietf.org/id/draft-ietf-http-range-retrieval-00.txt
list($range, $extra_ranges) = explode(',', $range_orig, 2);
}
else
{
$range = '';
header('HTTP/1.1 416 Requested Range Not Satisfiable');
exit;
}
}
else
{
$range = '';
}
//figure out download piece from range (if set)
list($seek_start, $seek_end) = explode('-', $range, 2);
//set start and end based on range (if set), else set defaults
//also check for invalid ranges.
$seek_end = (empty($seek_end)) ? ($file_size - 1) : min(abs(intval($seek_end)),($file_size - 1));
$seek_start = (empty($seek_start) || $seek_end < abs(intval($seek_start))) ? 0 : max(abs(intval($seek_start)),0);
//Only send partial content header if downloading a piece of the file (IE workaround)
if ($seek_start > 0 || $seek_end < ($file_size - 1))
{
header('HTTP/1.1 206 Partial Content');
header('Content-Range: bytes '.$seek_start.'-'.$seek_end.'/'.$file_size);
header('Content-Length: '.($seek_end - $seek_start + 1));
}
else
header("Content-Length: $file_size");
header('Accept-Ranges: bytes');
set_time_limit(0);
fseek($file, $seek_start);
while(!feof($file))
{
print(@fread($file, 1024*8));
ob_flush();
flush();
if (connection_status()!=0)
{
@fclose($file);
exit;
}
}
// file save was a success
@fclose($file);
exit;
}
else
{
// file couldn't be opened
header("HTTP/1.0 500 Internal Server Error");
exit;
}
}
else
{
// file does not exist
header("HTTP/1.0 404 File not found");
exit;
}
question from:
https://stackoverflow.com/questions/66051233/downloading-a-file-with-powershell-from-iis-express-server-with-php-randomly-sto 
