Is it possible that at moment you executed compare_gds all the Calibre licenses were in use?
Anyway, for two gds comparison it is better to use dbdiff utility.
The compare_gds utility compares two GDSII databases on files up to 2 GB in 32-bit mode and up to the internal memory limit in 64-bit mode. It does a flat layer-by-layer XOR (or NOT)operation, and writes an ASCII Calibre nmDRC results database.
The DBdiff application is a command-line-driven Calibre tool designed to compare hierarchy and shapes between two databases. This comparison is not meant to be a replacement for the standard XOR operation, but instead provides designers a way to find the differences between two databases quickly.
You can find full description of dbdiff command in Mentor documentation
I'm not sure about AND command usage in this utility but we switched to dbdiff several years ago because compare_gds limitations: it can run only flat, you need tricks to compare gds with datatypes etc..
In addition compare_gds tool is not fully supported by Mentor as almost any tool from :