The NetDB Web Service - A RESTful NetDB API
The NetDB web service is a RESTful API that makes it easy for NetDB users to write programs to find and manipulate NetDB data, and integrate NetDB with their systems in a independent and flexible manner.
Access Any NetDB account can use the NetDB web service. Authentication is via OAuth rather than Stanford's kerberos single sign-on service or Cardinal key, so an OAuth client ID must be associated with a NetDB account before it can use the NetDB web service. Instructions for requesting an OAuth client ID can be found here.It's possible to have a NetDB account with only OAuth / web service access. Such accounts can be requested using the instructions linked above. Mentioning that a web-service-only account is desired and no kerberos principal is needed will help the accounts creation team fulfill your request expeditiously.
Using OAuth Programming Environments There are three programming environments: the production environment, a development environment mirroring the production environment, and a pre-release environment with the next generation of features.Production Environment - netdb-api.stanford.edu The production environment comprises real, live, NetDB data. Changes can be monitored at netdb.stanford.edu. Changes made in the production environment are propagated to DNS, DHCP, and other downstream systems. Here is the OpenAPI documentation.Development Environment - netdb-dev-api.stanford.edu The purpose of the development environment is for NetDB users to develop and test their NetDB programs. It's a duplicate of the production NetDB and is refreshed Sunday at 4:42 AM using data from Saturday morning. Changes made in the development environment are never propagated to DNS, DHCP, or any downstream production system. Here is the OpenAPI documentation.There's a copy of the NetDB web application running on netdb-dev.stanford.edu that uses the same development database. Use it to set up data for testing and to examine the results of tests.
Early Access Development Environment - netdb-tng-api.stanford.edu The early access development environment is a pre-release environment where users can experiment with upcoming NetDB web service features. It uses the same database as the regular development environment, so netdb-dev.stanford.edu can be used to set up data for testing and to examine the results of tests. Here is the OpenAPI documentation.Efficient Programming - Working with Lists of Nodes Programs often need to make changes to a set of nodes based on some property or properties of the nodes. The simple solution is to get the nodes one by one, check the condition, and make the change if necessary. A slightly more complex and vastly more efficient solution is to perform a full search on all the nodes and conditions, then loop over the results making the changes. This method replaces the per-node call used to get the nodes with a single full search call. That can translate into tremendous performance improvements, especially when dealing with hundreds or thousands of nodes. Here are examples of the simple and efficient solutions.The efficient solution is accomplished with a search string of all the node names joined by or, e.g., argus.stanford.edu or avallone.stanford.edu or atalante.stanford.edu. As the number of elements in the search string increases the search slows down, slowly at first, but there's a critical point where the search slows markedly (as in orders of magnitude slower). Therefore, large sets, say, over 200 elements, should be broken into batches for maximal throughput. Here's an example of how to do that.
As to batch size, it depends somewhat on the length of the individual elements. Experiments in the development environment suggest a batch size between 100 and 200 is a good starting point. Of course testing with various batch sizes will yield the optimal batch size for a given application.
In summary, when dealing with sets of nodes, use full search and break large sets into batches of 100-200 using this algorithm.
Release History Mailing List A mailing list has been set up for NetDB web service users to ask questions and share ideas and programming experiences. It's netdb-web-service-users@lists.stanford.edu and you can join it here.Share and enjoy.
Updated: March 14, 2022