Cli-based utility for downloading VMware Products

Sometimes you think about uploading OVA images with VMware products directly into vCenter to make your life easier, or you're having an air-gapped environment, and you want to upload that distros images to file-transfer share.  To solve that problem, I was searching for a solution that could download the needed image from vExpert library.  At vExpert  Download page  I find a reference to VSM utility.

VSM reference at vExpert Download page

What is VSM?


VSM - smart VMware Software Manager for Linux systems, created by Edward Haletky (twitter). That utility provides an interface for downloading VMware products directly from VMware repository if you've got needed subscriptions and access to download products.

Installation


You can install the utility to Linux distro (Alpine / Debian-based / RedHat-based / PhotonOS / ArchLinux) via installer script. The utility is also available for local installation on macOS and Microsoft WSLv2. For Linux, you can download script locally to use only for user or system-wide. I really recommend you to install package from install.sh script (that will prevent a lot of problems with utility).

Here is a quick start via commands to install utility for system-wide usage:

wget https://raw.githubusercontent.com/Texiwill/aac-lib/master/vsm/install.sh
chmod 755 install.sh
./install.sh

Downloading distros


After installing the utility, you'll be able to use it not only for using regular VMware Customer Connect accounts, but to download vExpert packages as well (using active vExpert account) with command:

/usr/local/bin/vsm.sh -y --vexpertx --historical --patches --dts --oem  --symlink --compress

Additional info about using the utility and how to work with vExpert Library available here.

After successful login using your VMware and vExpert accounts, you'll get access to all product sections that contains VMware distros that are available for your accounts:

akoznov@ubnt-lab01:~/bin$ /usr/local/bin/vsm.sh -y --vexpertx --historical --patches --dts --oem --symlink --compress
Using the following options:
        Version:        6.7.7
        OS Mode:        ubuntu
        VSM XML Dir:    /tmp/vsm.akoznov
        Repo Dir:       /tmp/vsm.akoznov
        Dryrun:         0
        Rebuild:        0
        Force Download: 0
        Checksum:       0
        Historical Mode:1
        Symlink Mode:   1
        Reset XML Dir:  0
        Nested: 0
        Verify Mode:    0
        Retry Count:    8
        Use credstore:  1
        Saving to /home/akoznov/.vsmrc
        Authenticating... May take up to 90s... 12-20s is normal...
? Enter MFA pass code:  XXXXXX
        Oauth:          1
        Patches:        1
/tmp/vsm.akoznov/depot.vmware.com/PROD/channel/_h_downloads.xhtml               100%[====================================================================================================================================================================================================>]  44,90K  --.-KB/s    in 0,03s
        Use credstore:  1
 1) Datacenter_&_Cloud_Infrastructure
 2) Infrastructure_&_Operations_Management
 3) Networking_&_Security
 4) Infrastructure-as-a-Service
 5) Desktop_&_End-User_Computing
 6) Endpoint_&_Workload_Security
 7) Internet_of_things_[IOT]
 8) Cloud_Services
 9) Other
10) VMware_Blockchain
11) VMware_Tanzu_Data_Services
12) Exit
#?

Via manual navigation through menu, you can download needed files. For example, if you want to download latest NSX upgrade bundle for version 4.0.0.1 (VMware-NSX-upgrade-bundle-4.0.0.1.0.20159689.mub), you should use numbers 3-1-1-1-4-4-13:

 1) Datacenter_&_Cloud_Infrastructure
 2) Infrastructure_&_Operations_Management
 3) Networking_&_Security
 4) Infrastructure-as-a-Service
 5) Desktop_&_End-User_Computing
 6) Endpoint_&_Workload_Security
 7) Internet_of_things_[IOT]
 8) Cloud_Services
 9) Other
10) VMware_Blockchain
11) VMware_Tanzu_Data_Services
12) Exit
#? 3
 1) VMware_NSX
 2) VMware_NSX-T_Data_Center
 3) VMware_NSX_Data_Center_for_vSphere
 4) VMWare_NSX_Intelligence
 5) VMware_SD-WAN
 6) VMware_NSX_Security
 7) VMware_NSX_Advanced_Load_Balancer
 8) VMware_Global_Network_Identities
 9) VMware_Antrea
10) NSX_Network_Detection_and_Response
11) Back
12) Exit
#? 1
1) VMware_NSX_4_x
2) Back
3) Exit
#? 1
1) VMware_NSX_4_x_VMware_NSX_Standard
2) VMware_NSX_4_x_VMware_NSX_Professional
3) VMware_NSX_4_x_VMware_NSX__Advanced
4) VMware_NSX_4_x_VMware_NSX_Enterprise_Plus
5) VMware_NSX_4_x_VMware_Network_Management_add-on
6) Back
7) Exit
#? 1
1) All
2) VIDM_ONPREM_3360
3) VRLI_8100_NSX
4) NSX_4011
5) Mark
6) Back
7) Exit
#? 4
1) NSX_4011_LE
2) NSX_4011
3) NSX_4001_LE
4) NSX_4001
5) Back
6) Exit
#? 4
/tmp/vsm.akoznov/depot.vmware.com/PROD/channel/_NSX_4001_ver.xhtml              100%[====================================================================================================================================================================================================>]   2,04K  --.-KB/s    in 0s
/tmp/vsm.akoznov/depot.vmware.com/PROD/channel/__NSX_4001.xhtml                 100%[====================================================================================================================================================================================================>]  23,42K  --.-KB/s    in 0,03s
/tmp/vsm.akoznov/depot.vmware.com/PROD/channel/_NSX_4001_dts.xhtml              100%[====================================================================================================================================================================================================>]   2,05K  --.-KB/s    in 0s
/tmp/vsm.akoznov/depot.vmware.com/PROD/channel/_NSX_4001_oem.xhtml              100%[====================================================================================================================================================================================================>]      23  --.-KB/s    in 0s
/tmp/vsm.akoznov/depot.vmware.com/PROD/channel/_NSX_4001_add.xhtml              100%[====================================================================================================================================================================================================>]      23  --.-KB/s    in 0s
 1) All
 2) nsx-unified-appliance-4.0.0.1.0.20159694.ova
 3) nsx-embedded-unified-appliance-4.0.0.1.0.20159694.ova
 4) VMware-NSX-Application-Platform-3.2.1.1.0.20140674.tgz
 5) kubernetes-tools-1.18.20-00_3.5.4-1.tar.gz
 6) kubernetes-tools-1.20.11-00_3.5.4-1.tar.gz
 7) VMware-NSX-Malware-Prevention-appliance-3.2.1.0.0.19801960.ova
 8) nsx-edge-4.0.0.1.0.20159697.iso
 9) nsx-edge-4.0.0.1.0.20159697.ova
10) nsx-lcp-4.0.0.1.0.20159692-esx70.zip
11) nsx-l2vpn-client-ovf-11197779.tar.gz
12) nsx-data-migration-dry-run-4.0.0.1.0.20160804.ova
13) VMware-NSX-upgrade-bundle-4.0.0.1.0.20159689.mub
14) VMware-CC-upgrade-bundle-4.0.0.1.0.20159689.mub
15) VMware-NSX-edge-4.0.0.1.0.20159697.nub
16) nsx-lcp-4.0.0.1.0.20159692-baremetal-server-rhel76_x86_64.tar.gz
17) nsx-lcp-4.0.0.1.0.20159692-baremetal-container-rhel76_x86_64.tar.gz
18) nsx-lcp-4.0.0.1.0.20159692-baremetal-server-rhel77_x86_64.tar.gz
19) nsx-lcp-4.0.0.1.0.20159692-baremetal-container-rhel77_x86_64.tar.gz
20) nsx-lcp-4.0.0.1.0.20159692-baremetal-server-rhel78_x86_64.tar.gz
21) nsx-lcp-4.0.0.1.0.20159692-baremetal-container-rhel78_x86_64.tar.gz
22) nsx-lcp-4.0.0.1.0.20159692-baremetal-server-rhel79_x86_64.tar.gz
23) nsx-lcp-4.0.0.1.0.20159692-baremetal-container-rhel79_x86_64.tar.gz
24) nsx-lcp-4.0.0.1.0.20159692-baremetal-server-rhel80_x86_64.tar.gz
25) nsx-lcp-4.0.0.1.0.20159692-baremetal-server-rhel83_x86_64.tar.gz
26) nsx-lcp-4.0.0.1.0.20159692-baremetal-server-linux64-sles12sp3.tar.gz
27) nsx-lcp-4.0.0.1.0.20159692-baremetal-server-linux64-sles12sp4.tar.gz
28) nsx-lcp-4.0.0.20159692-baremetal-server-win32_vs2017.zip
29) DriversTools
30) Back
31) Exit
#? 13
Downloading VMware-NSX-upgrade-bundle-4.0.0.1.0.20159689.mub to /tmp/vsm.akoznov/dlg_NSX_4001:
VMware-NSX-upgrade-bundle-4.0.0.1.0.20159689.mub                                  4%[=======>                                                                                                                                                                                             ] 378,59M  10,5MB/s

By default, the utility will download files to a directory:

/tmp/vsm.username/dlg_something/

Where username - Linux account name that was used to run vsm.sh, something - is product name that will contain downloaded files. In our case, distro of NSX will be placed to path:

/tmp/vsm.akoznov/dlg_NSX_4001/VMware-NSX-upgrade-bundle-4.0.0.1.0.20159689.mub

Searching files


By default, the utility downloads the file newlocs.json, and it contains vCenter and ESXi bits of packages and info about them. If we check contents for that file after installing the utility, we'll see only packages for that 2 categories (190Kb file with 193730 symbols in it for utility version 6.7.7). We can find ZIP depots for 7.0.X and 8.0 versions that is mentioned in that JSON scheme:

akoznov@ubnt-lab01: less /tmp/vsm.akoznov/depot.vmware.com/PROD/channel/newlocs.json

akoznov@ubnt-lab01:/tmp/vsm.akoznov/depot.vmware.com/PROD/channel$ wc -m newlocs.json
193730 newlocs.json


akoznov@ubnt-lab01:/tmp/vsm.akoznov/depot.vmware.com/PROD/channel$ cat newlocs.json | jq | grep 'VMware-ESXi-[7-8].*.[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]-depot.zip'
      "name": "VMware-ESXi-7.0.0-15843807-depot.zip",
      "name": "VMware-ESXi-7.0U1-16850804-depot.zip",
      "name": "VMware-ESXi-7.0U1c-17325551-depot.zip",
      "name": "VMware-ESXi-7.0U2a-17867351-depot.zip",
      "name": "VMware-ESXi-7.0U3c-19193900-depot.zip",
      "name": "VMware-ESXi-7.0U3d-19482537-depot.zip",
      "name": "VMware-ESXi-7.0U3f-20036589-depot.zip",
      "name": "VMware-ESXi-7.0U3g-20328353-depot.zip",
      "name": "VMware-ESXi-7.0b-16324942-depot.zip",
      "name": "VMware-ESXi-8.0-20513097-depot.zip",

If we want to download a file that is listed in the JSON cache, you can use command:

/usr/local/bin/vsm.sh -y --dlg VMware-ESXi-8.0-20513097-depot.zip

If you want to download files that are not listed in JSON scheme, at first you should manually run vsm.sh with rebuild and dryrun keys, navigate through needed categories to folders that are contains distros. After that, you'll have an updated cache that will contain info about locations of that files at VMware repositories and only after that, you can use dlg keys to download non-default files. In our example, we've already navigated through the NSX section, and we can see that newlocs.json contains info about NSX distros:

/usr/local/bin/vsm.sh --rebuild --dryrun

akoznov@ubnt-lab01:/tmp/vsm.akoznov/depot.vmware.com/PROD/channel$ wc -m newlocs.json
204212 newlocs.json

akoznov@ubnt-lab01:/tmp/vsm.akoznov/depot.vmware.com/PROD/channel$ cat newlocs.json | jq | grep 'nsx-embedded-unified-appliance-'
      "name": "nsx-embedded-unified-appliance-4.0.0.1.0.20159694.ova",
      "name": "nsx-embedded-unified-appliance-4.0.1.1.0.20598732.ova",
      
/usr/local/bin/vsm.sh -y --dlg nsx-embedded-unified-appliance-4.0.1.1.0.20598732.ova
Using the following options:
        Version:        6.7.7
        OS Mode:        ubuntu
        VSM XML Dir:    /tmp/vsm.akoznov
        Repo Dir:       /tmp/vsm.akoznov
        Dryrun:         0
        Rebuild:        0
        Force Download: 0
        Checksum:       0
        Historical Mode:1
        Symlink Mode:   1
        Reset XML Dir:  0
        Nested: 0
        Verify Mode:    0
        Retry Count:    8
        Use credstore:  1
        Saving to /home/akoznov/.vsmrc
        Oauth:          1
        Patches:        1
        Use credstore:  1
Downloading nsx-embedded-unified-appliance-4.0.1.1.0.20598732.ova to /tmp/vsm.akoznov/dlg_NSX_4011:
nsx-embedded-unified-appliance-4.0.1.1.0.20598732.ova                             0%[  ] 28,65M  9,61MB/s

I hope that article will make your life easy and that utility might be helpful in your automations or using distros in air-gapped environments =)