You signed in with another tab or window.Reloadto refresh your session.You signed out in another tab or window.Reloadto refresh your session.You switched accounts on another tab or window.Reloadto refresh your session.Dismiss alert
This folder contains aBackground Functionwhich deletes DNS A recordswhen a VM is deleted.
Please noteDNS record deletion is implemented, however, cannot beguaranteed. A race exists between the function obtaining the VM IP address andthecompute.instances.deleteoperation. If the VM is deleted before the IPis obtained, the function will not delete the DNS record because it cannotcheck the IP address matches the VM being deleted.
In practice this background function collects the IP address well within the~30 second window of the VM delete operation.
Structured logs enable VM deletions which were not processed because the racewas lost. SeeLost Racefor log filters to identify VM's deletedbefore cleanup could take place.
Project Setup
This example has been developed for use with multiple service projects. Acentralized logs project is used to host one pubsub topic for all VM deletionevents. One deployment of the function implements the event handler.
The logs project contains the dns-vm-gc Pub/Sub topic and thedns_vm_gc function deployed as a Background Function.
One or more service projects contain VM resources to be deleted.
The host project contains a VPC shared with the user project and DNSresource record sets needing to be cleaned up automatically.
Identify the Logs Project
Identify a project to host thevm-deletionsPub/Sub topic and the DNS VM GCCloud Function. Service projects are configured to export filtered logs intothis topic.
If a project does not already exist, create a new project. A suggested name islogs. The rest of this document will uselogs-123456as the project ID forthe centralized logs project.
Create the vm-deletions Pub/Sub topic
Service projects exportcompute.instances.deleteevents to thevm-deletionstopic. The VM DNS GC background function subscribes to this topic and triggerson each event.
Create a topic namedvm-deletionsin the logs project as perCreate atopic.
Configure Log Exports
Configure Log Exports in one or more service projects. Logs are exported tothevm-deletionstopic in the logs project.
Stackdriver logs exportsare used to convey VM lifecycle eventsto the DNS VM GC function via Cloud Pub/Sub. A Stackdriver filter is used tolimit logs to VM deletion events, reducing data traveling through Pub/Sub.
Configure an export to thevm-deletionstopic with the following filter, forexampleprojects/logs-123456/topics/vm-deletions.
This filter results in one event published per VM deletion, aGCE_API_CALLevent when the VM deletion is requested.
If additional events are published to the topic, the function triggers, butignores events which do not match this filter.
Service Account
The Background Function runs with a service account identity. Create a serviceaccount nameddns-vm-gcin the logs project for this purpose. This exampleassumesGCP-managedkeys.
If you are modifying this example you may download the service account key andrun locally as the service account using the GOOGLE_APPLICATION_CREDENTIALSenvironment file. SeeProviding credentials to your applicationfordetails.
Service Account Roles
The Background Function service account requires the following roles.
DNS Admin
Grant the DNS Admin role to the dns-vm-gc service account in the host project.DNS Admin allows the DNS VM GC function to delete DNS records in the hostproject.
This role may be granted at the Shared VPC project level.
Compute Viewer
Grant the Compute Viewer role to the dns-vm-gc service account. Compute Viewerallows the DNS VM GC function to read the IP address of the VM, necessary toensure the correct A record is deleted.
This role may be granted at the project, folder or organization level asappropriate.
Logs Writer
Grant the Logs Writer role to the dns-vm-gc service account. Logs Writer isrequired to write structured event logs to theReportingStream.
This role may be granted at the project, folder, or organization level asappropriate. It is recommended to grant the role at the same level the logstream exists at, the logging project by default. SeeCustom ReportingDestinationfor more information.
Deployment
Deploy this function into the logs project to simplify the subscription to thevm-deletionstopic.
Environment variables are used to configure the behavior of the function.Update the env.yaml file to reflect the correct VPC Host project and ManagedZone names for your environment. A sample is provided in env.yaml.sample.
The DNS VM GC function logs into two different locations. Structured Eventsintended for reporting are sent to a special purpose reporting stream. Plaintext logs are sent to the standard Cloud Function logs accessible viagcloud functions logs read.
Reporting Stream
The reporting stream is intended to answer two primary questions:
Which VM deletion events, if any, were not processed?
What records were deleted automatically?
When the function loses the race against the delete operation, the event is notprocessed and the function reports a detail code ofLOST_RACE.
When the function deletes a record automatically, the fully qualified domainname is logged along with a detail code ofRR_DELETEDfor resource recorddeleted.
Custom Reporting Destination
By default the reporting stream is located atprojects/<logs_project>/logs/<function_name>. The reporting stream isconfigurable by setting theDNS_VM_GC_REPORTING_LOG_STREAMenvironmentvariable when deploying the function. For example, to send reporting events tothe organization level:
The function also logs unstructured plain text logs usingCloud FunctionLogs. Becasue these logs are unstructured, they are less usefulthan the Report Stream logs for reporting purposes, however, are present tokeep all activity associated together with each execution ID of the function.
Note the cloud function logs have an execution_id. This execution ID is notreadily available at runtime and therefore absent from the structured reportlog stream. The function logs a message with theevent_idbeing processed toassociate the execution_id with the event_id. This behavior is intended tocorrelate each execution in the Cloud Function Logs with each report in theReport Stream. The correlation of execution_id to event_id is not necessaryfor day to day reporting. The correlation is useful for the rare situation ofcomplete end-to-end tracing.
Reporting
Lost Race
Periodic reporting should be performed to monitor forNOT_PROCESSEDresults.In the event of a lost race, automatic DNS record deletion is not guaranteed.
The following Stackdriver Advanced Filter identifies when a VM deletion eventwas not processed automatically:
Debug logs are also available, but are not sent by default. To enable, deploythe function with theDEBUGenvironment variable set to a non-empty string.Note, debug logs generates 2*N log events every time a VM is deleted where N isthe number of DNS records across all configured managed zones. For example,deleting 10 VM instances with 1,000 managed DNS records generates 20,000 debuglog entries at minimum.
Detail Codes
The following detail codes may be reported to the reporting stream:
Detail Code
Description
Result
NO_MATCHES
No DNS records matched the VM deleted
OK
RR_DELETED
A DNS record matched and has been deleted
OK
VM_NO_IP
The function won the race, but the VM has no IP
OK
IGNORED_EVENT
Trigger event is not a VM delete GCE_API_CALL
OK
LOST_RACE
The VM was deleted before the IP was determined
NOT_PROCESSED
In addition, there are detail codes when DEBUG is turned on indicating thereason why DNS records were not automatically deleted.