Windows下unzip解壓縮遇到的warning: xxx appears to use backslashes as path separators

IBM Spectrum Symphony是基於SOA架構的分佈式計算框架,它能在自由伸縮的共享集羣中,爲計算密集型和數據密集型的應用提供強大的企業級管理。

Symphony有一個配置是這樣的,在應用的profile裏頭,有個方法是Register,可以配置成actionOnSI="blockHost"。這樣當某service實例在Register方法中由於某種原因(比方說磁盤不足、單點故障等)而失敗的話,計算節點就會被阻止加入到計算,那麼Symphony會從資源組裏分配其他可用的計算節點參與運算。這樣做的目的是避免作業一直分配到某個不能成功完成運算的節點,導致時間浪費甚至作業得不到計算。

      <Method name="Register">
        <Timeout actionOnSI="blockHost" duration="600"/>
        <Exit actionOnSI="blockHost"/>
      </Method>

 

某天,客戶遇到這樣一個問題。他們更新了應用的service代碼,要部署到cluster。部署是成功的,但是在解壓縮的時候,出錯了,結果就是節點被blocked。爲了方便理解,先貼上Symphony SOAM這部分的架構圖。紅框部分是Symphony中間件,客戶僅需要開發客戶端client和服務端service,client用於提交作業,service實例用於計算作業,至於資源調度和作業調度則由中間件完成,客戶不需要考慮。

所以我們需要看SIM(圖中的Service Instance Manager)的日誌,因爲是SIM管理着SI(圖中的Service Instance),包括SI的啓動、更新、退出等。下面是SIM的日誌

 15:51:06.951 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::copyArchive: Copying file C:\Windows\TEMP\\SI_Package_Name_Version\SI_Package_Name_Version.zip to D:\grid\ibm/soam\deploy\SI_Package_Name_Version.v1\SI_Package_Name_Version.zip

 15:51:06.966 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::execNextStep: Executing uncompress or validate step: D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\bin\..\bin\utils\unzip -o -q "SI_Package_Name_Version.zip"

 15:51:06.966 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::execCmd: Execute PATH: D:\grid\ibm/soam/7.2/w2k3_x64-vc7-psdk/lib64;D:\grid\ibm/soam/7.2/w2k3_x64-vc7-psdk/bin;D:\grid\ibm/soam/7.2/w2k3_x64-vc7-psdk/lib;D:\grid\ibm\3.6\lib;D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\bin;D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\lib;D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\lib64;D:\grid\ibm\3.6\bin;D:\grid\ibm\3.6\lib;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\\Hewlett-Packard\HPSUT\bin;C:\Program Files\HP\HP BTO Software\lib;C:\Program Files\HP\HP BTO Software\bin;C:\Program Files\HP\HP BTO Software\bin\win64;C:\Program Files\HP\HP BTO Software\bin\win64\OpC;;D:\grid\ibm\3.6\lib;D:\grid\ibm\3.6\bin

 15:51:06.966 GMT DEBUG [5268:17244] CLI.dpl.download.RunTask - RunTask::execBlock(): Started: D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\bin\..\bin\utils\unzip -o -q "SI_Package_Name_Version.zip" with tid 388881

 15:51:07.869 GMT DEBUG [5268:17244] CLI.dpl.download.RunTask - RunTask::execBlock(): There may be some exceptions or errors during decompressing process, and the status of the process is 1

 15:51:07.869 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::execNextStep: Decompress or validate command failed.

 15:51:07.869 GMT INFO [5268:17244] CLI.dpl.deployutils - warning: SI_Package_Name_Version.zip appears to use backslashes as path separators

 15:51:07.869 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::extract: Deleting file D:\grid\ibm/soam\deploy\SI_Package_Name_Version.v1\SI_Package_Name_Version.zip 

 15:51:07.869 GMT DEBUG [5268:17244] CLI.dpl.download.InstallModule - InstallModule::copyModuleAndDecompress(): Failed to uncompress: C:\Windows\TEMP\\SI_Package_Name_Version\SI_Package_Name_Version.zip

 15:51:07.869 GMT ERROR [5268:17244] CLI.dpl.download.InstallModule - Domain <Application>: Failed to uncompress the package, C:\Windows\TEMP\\SI_Package_Name_Version\SI_Package_Name_Version.zip. Check if the de-archive utility is available or there is enough disk storage for uncompressing.

 15:51:07.916 GMT DEBUG [5268:17244] CLI.SoamDeploy.CliSoamDeployDownloadImpl - Deleting tmp file: C:\Windows\TEMP\\SI_Package_Name_Version\SI_Package_Name_Version.zip

 15:51:07.916 GMT ERROR [5268:17244] CLI.SoamDeploy.CliSoamDeployDownloadImpl - Domain <Application>: The package <SI_Package_Name_Version> failed to install. The deployed package file could not be decompressed. Check if: (1) the decompression utility is available, (2) there is enough disk storage for decompressing the file, or (3) the package file is too large to be decompressed.

從SIM日誌來看,新的SI的package是已經下載到本地了,但是在解壓縮的時候,報了個warning級別的消息:"SI_Package_Name_Version.zip appears to use backslashes as path separators",解壓縮的命令是unzip -o -q "SI_Package_Name_Version.zip"。根據後邊的日誌,可能的原因包括:

(1) unzip工具不存在;

(2) 磁盤空間不足;

(3) 包太大。

當然,客戶檢查下來,這三個懷疑點都不是問題。

 

那會不會是壓縮包本身有問題呢?於是讓客戶在同一臺計算節點上,手動去解壓縮那個包。果然,手動解壓縮也有類似的warning,只不過雖然報了這個warning,解壓縮是成功的,所有文件和目錄都解壓出來了。

D:\grid\ibm\soam\deploy\Applications\SampleAppJava\SI_Package_Name_Version.v1

>D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\bin\..\bin\utils\unzip -o -q SI_Package_Name_Version.zip

warning: SI_Package_Name_Version.zip appears to use backslashes as path separators

至此,可以斷定跟Symphony關係不大,要麼是壓縮包有問題,要麼是unzip不能解壓縮這個壓縮包。

 

通過詢問客戶得知,客戶的壓縮文件都是經過Windows的Powershell壓縮工具Compress-Archive生成的。於是模仿客戶的包的格式,在實驗室裏可以復現相同問題。原來問題在於,使用Compress-Archive壓縮帶有文件夾的內容時,壓縮沒有問題;但用unzip去解壓縮時,就會遇到“use backslashes as path separators”問題。進一步測試發現,在Windows中如果通過右擊內容,選擇"Compress(zipped) folder"的方式,或者通過7-zip來壓縮的方式,unzip都可以完美解壓。

PS C:\Users\Administrator> Compress-Archive

cmdlet Compress-Archive at command pipeline position 1

Supply values for the following parameters:

Path[0]: C:\Users\Administrator\file1.log

Path[1]: C:\Users\Administrator\lab\

Path[2]:

DestinationPath: C:\Users\Administrator\package.zip

PS C:\Users\Administrator> C:\symphony\soam\7.3\w2k3_x64-vc7-psdk\bin\utils\unzip.exe -o -q .\package.zip

warning: ./package.zip appears to use backslashes as path separators

PS C:\Users\Administrator> C:\symphony\soam\7.3\w2k3_x64-vc7-psdk\bin\utils\unzip.exe -o -q .\package.zip

warning: ./package.zip appears to use backslashes as path separators

PS C:\Users\Administrator> Compress-Archive

所以,嚴格地說,這不是Symphony的問題,是Windows的Compress-Archive工具的問題。一頓Google,果然找到Powershell報告這是一個Microsoft.PowerShell.Archive的bug,在版本1.2.3.0得到修復,詳情請參考https://github.com/PowerShell/Microsoft.PowerShell.Archive/issues/48

由於客戶堅持要用Microsoft.PowerShell.Archive去壓縮,所以解決方法只能是升級Microsoft.PowerShell.Archive,方法請參考https://www.powershellgallery.com/packages/Microsoft.PowerShell.Archive/1.2.3.0

 

下面是我在實驗室裏升級Microsoft.PowerShell.Archive和驗證問題得到解決的操作:

PS C:\Users\egoadmin> Install-Module -Name Microsoft.PowerShell.Archive -RequiredVersion 1.2.3.0 

NuGet provider is required to continue

PowerShellGet requires NuGet provider version '2.8.5.201' or newer to interact with NuGet-based repositories. The NuGet

provider must be available in 'C:\Program Files\PackageManagement\ProviderAssemblies' or

'C:\Users\egoadmin\AppData\Local\PackageManagement\ProviderAssemblies'. You can also install the NuGet provider by

running 'Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force'. Do you want PowerShellGet to install

and import the NuGet provider now?

[Y] Yes [N] No [S] Suspend [?] Help (default is "Y"): y

Untrusted repository

You are installing the modules from an untrusted repository. If you trust this repository, change its

InstallationPolicy value by running the Set-PSRepository cmdlet. Are you sure you want to install the modules from

'PSGallery'?

[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): A

PS C:\Users\egoadmin>

PS C:\Users\egoadmin>

PS C:\Users\egoadmin> Get-Module -ListAvailable -Name Microsoft.PowerShell.Archive

Directory: C:\Program Files\WindowsPowerShell\Modules

ModuleType Version Name ExportedCommands

---------- ------- ---- ----------------

Manifest 1.2.3.0 Microsoft.PowerShell.Archive {Compress-Archive, Expand-Archive}

Directory: C:\windows\system32\WindowsPowerShell\v1.0\Modules

 

ModuleType Version Name ExportedCommands

---------- ------- ---- ----------------

Manifest 1.0.1.0 Microsoft.PowerShell.Archive {Compress-Archive, Expand-Archive}

PS C:\Users\egoadmin\Downloads> dir

Directory: C:\Users\egoadmin\Downloads

Mode LastWriteTime Length Name

---- ------------- ------ ----

d----- 6/1/2020 7:53 AM lab

d----- 6/1/2020 7:42 AM unzip552

-a---- 6/1/2020 7:53 AM 38 1234.txt

-a---- 6/1/2020 7:38 AM 1295576 ChromeSetup.exe

-a---- 6/1/2020 7:40 AM 1305781 unzip552.zip

PS C:\Users\egoadmin\Downloads> Compress-Archive 

cmdlet Compress-Archive at command pipeline position 1

Supply values for the following parameters:

Path[0]: lab

Path[1]: 1234.txt

Path[2]:

DestinationPath: 1.zip

PS C:\Users\egoadmin\Downloads> dir

Directory: C:\Users\egoadmin\Downloads

Mode LastWriteTime Length Name

---- ------------- ------ ----

d----- 6/1/2020 7:53 AM lab

d----- 6/1/2020 7:42 AM unzip552

-a---- 6/1/2020 8:03 AM 290 1.zip

-a---- 6/1/2020 7:53 AM 38 1234.txt

-a---- 6/1/2020 7:38 AM 1295576 ChromeSetup.exe

-a---- 6/1/2020 7:40 AM 1305781 unzip552.zip

C:\Users\Administrator\Downloads>E:\SpectrumComputing\soam\7.2\w2k3_x64-vc7-psdk\bin\utils\unzip.exe -o -q 1.zip 

C:\Users\Administrator\Downloads>dir

Volume in drive C has no label.

Volume Serial Number is 74B6-4C2C

Directory of C:\Users\Administrator\Downloads

06/01/2020 12:54 AM <DIR> .

06/01/2020 12:54 AM <DIR> ..

06/01/2020 01:06 AM 290 1.zip

06/01/2020 07:53 AM 38 1234.txt

06/01/2020 12:54 AM <DIR> lab

2 File(s) 328 bytes

3 Dir(s) 4,173,824 bytes free

 

 

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章