Unless otherwise noted, changes described below apply to the newest Chrome beta channel release for Android, Chrome OS, Linux, macOS, and Windows. Learn more about the features listed here through the provided links or from the list on ChromeStatus.com. Chrome 87 is beta as of October 15, 2020.
WebAuthn Tab in DevTools
Testing web authentication has long been difficult because developers need devices to test their code. Starting in Chrome 87, authentication can be emulated and debugged using a new panel in DevTools. You can find the panel in DevTools by selecting More options, then More tools, then WebAuthn. To learn how to use it, see the section in What’s New in DevTools (Chrome 87).
Control camera pan, tilt, and zoom
Room-scale video conferencing solutions deploy cameras with pan, tilt, and zoom capabilities so that software can point the camera at meeting participants. Starting in Chrome 87, the pan, tilt, and zoom features on cameras are accessible to websites using media track constraints in MediaDevices.getUserMedia() and MediaStreamTrack.applyConstraints().
Websites are only allowed to control these capabilities when users explicitly grant permission. For details on using the new capabilities and a demo, see Control camera pan, tilt, and zoom.
CSS flow-relative shorthand and offset properties
The trend in CSS for many years has been to supplement physical properties with logical properties. Properties that assume language flows left to right and top to bottom don’t work in non-European text such as vertical Chinese text, or Arabic. Modern CSS rules use flow-relative terms like start and end and provide rules for dealing with the text’s axis (direction).
The first step in implementing this in Chrome was to implement the most granular flow-relative features of the CSS Logical Properties and Values spec. Chrome 87 ships shorthands and
offsets to make these logical properties and values a bit easier to write. What was once written with multiple CSS rules can now be written as one. For example, separate rules for margin-block-start and margin-block-end may now be written using a single margin-block property.
For a list of all flow-relative shorthands now supported by Chrome, and explanations for how to use them, see Logical layout enhancements with flow-relative shorthands. For more CSS-related updates, see the CSS section, below.
Completed Origin Trials
Origin trials allow you to try new features and give feedback on usability, practicality, and effectiveness to the web standards community. The following feature, previously in a Chrome origin trial, is now enabled by default.
Cookie Store API
The Cookie Store API exposes HTTP cookies to service workers and offers an asynchronous alternative to document.cookie.
Other features in this release
cross-origin isolation
Chrome 87 has a number of changes related to cross-origin isolation. Chrome will now use origin instead of site as agent cluster key for cross-origin isolated agent clusters. Mutation of document.domain is no longer supported for cross-origin isolated agent clusters. This change also introduces window.crossOriginIsolated, a boolean that indicates whether APIs that require cross-origin isolation are allowed to use it. Supporting APIs include:
SharedArrayBuffer(required for WebAssembly Threads)performance.measureMemory()- JS Self-Profiling API
For more information, see Making your website “cross-origin isolated” using COOP and COEP.
iframe attribute for limiting same-origin iframe document access
Adds the disallowdocumentaccess property to disallow cross-document scripting between iframes from the same origin in the same parent document. This also puts same-origin iframes in separate event loops.
isInputPending()
Sometimes long-running scripts block user input. A lag between a user’s action and a response by an app is a bad user experience. To address this, Chrome has added a method called isInputPending(), accessible from navigator.scheduling, which can be called from long-running operations. You can find an example of the method’s use in the draft spec.
Range Request Headers in Service Workers
HTTP range requests, which have been available in major browsers for several years, allow servers to send requested data to the client in chunks. This has proved especially useful for large media files where the user experience is improved through smoother playback and improved pause and resume functions.
Historically, range requests and services workers did not work well together, forcing developers to build work-arounds. Starting in Chrome 87, passing range requests through to the network from inside a service worker will “just work.”
For an explanation of the issues with range requests and what’s changed in Chrome 87, see Handling range requests in a service worker.
Streams API: transferable streams
Transferable streams now allows ReadableStream, WritableStream, and TransformStream objects to be passed as arguments to postMessage(). The streams APIs provide ubiquitous, interoperable primitives for creating, composing, and consuming streams of data. A natural thing to do with a stream is to pass it to a web worker. This provides a fluent primitive for offloading work to another thread.
Offloading work onto a worker is important for a smooth user experience, but the ergonomics can be awkward. Transferable streams solve this problem for streams. Once the stream itself has been transferred, the data is transparently cloned in the background.
Transition related event handlers
The ontransitionrun, ontransitionstart, and ontransitioncancel event handler attributes allow developers to add event listeners for 'transitionrun', 'transitionstart', and 'transitioncancel' events on elements, Document objects, and Window objects.
WakeLockSentinel.released Attribute
The WakeLockSentinel object has a new property called released that indicates whether a sentinel has already been released. It defaults to false and changes to true when a release event is dispatched. The new attribute helps web developers know when locks are released so that they do not need to keep track of them manually.
CSS
@font-face descriptors to override font metrics
New @font-face descriptors have been added to ascent-override, descent-override, and line-gap-override to override metrics of the font. This Improves interoperably across browsers and operating systems, so that the same font always looks the same on the same site, regardless of OS or browser. Additionally, it aligns metrics between two web fonts present simultaneously, but for different glyphs. Finally, it overrides font metrics for a fallback font to emulate a web font, to minimize cumulative layout shift.
Text Decoration and Underline Properties
Chrome now supports several new text decoration and underline properties. These properties solve use cases where underlines are too close to the text baseline and ink-skipping triggers too early in a text run. These use cases solve problems caused by the launch of the text-decoration-skip-ink property. The new properties are text-decoration-thickness, text-underline-offset and a from-font keyword for text-underline-position.
The quotes Property Supports the ‘auto’ Value
CSS2 allowed browsers to define the default value for the quotes property, which Chrome formerly followed. Chrome 87 now follows CSS Generated Content Module Level 3 in which the 'auto' keyword is the default value. That spec requires that a typographically appropriate value be used for quotes based on the content language of the element and/or its parent.
JavaScript
This version of Chrome incorporates version 8.7 of the V8 JavaScript engine. It specifically includes the changes listed below. You can find a complete list of recent features in the V8 release notes.
Atomics.waitAsync()
Chrome now supports Atomics.waitAsync(), the async version of Atomics.wait(). Atomics.waitAsync() allows programmers to wait on a SharedArrayBuffer location in the same fashion as Atomics.wait() but returns a Promise instead.
Atomics.wait() blocks the thread and cannot be used on the main web browser thread, where blocking is disallowed. This makes coordination via SharedArrayBuffers between the main thread and worker threads more ergonomic.
Deprecations and Removals
This version of Chrome introduces the deprecations and removals listed below. Visit ChromeStatus.com for lists of current deprecations and previous removals.
Comma separator in iframe allow attribute
Permissions policy declarations in an <iframe> tag can no longer use commas as a separator between items. Developers should use semicolons instead.
-webkit-font-size-delta
Blink will no longer support the rarely-used -webkit-font-size-delta property. Developers should use font-size to control font size instead.
Deprecate FTP support
Chrome is deprecating and removing support for FTP URLs. The current FTP implementation in Google Chrome has no support for encrypted connections (FTPS), nor proxies. Usage of FTP in the browser is sufficiently low that it is no longer viable to invest in improving the existing FTP client. In addition, more capable FTP clients are available on all affected platforms.
Google Chrome 72 and later removed support for fetching document subresources over FTP and rendering of top level FTP resources. Currently navigating to FTP URLs results in showing a directory listing or a download depending on the type of resource. A bug in Google Chrome 74 and later resulted in dropping support for accessing FTP URLs over HTTP proxies. Proxy support for FTP was removed entirely in Google Chrome 76. In Chrome 86, FTP was turned off for pre-release channels (Canary and Beta) and was experimentally turned off for one percent of stable users.
The remaining capabilities of Google Chrome’s FTP implementation are restricted to either displaying a directory listing or downloading a resource over unencrypted connections.
Remainder of the deprecation follows this timeline:
Chrome 87
FTP support will be disabled by default for fifty percent of users but can be enabled using the flags listed above.
Chrome 88
FTP support will be disabled.
Open source software is the foundation of many modern software products. Over the years, developers increasingly have relied on reusable open source components for their applications. It is paramount that these open source components are secure and reliable, as weaknesses impact those that build upon it.
Google cares deeply about the security of the open source ecosystem and recently launched the Open Source Security Foundation with other industry partners. Fuzzing is an automated testing technique to find bugs by feeding unexpected inputs to a target program. At Google, we leverage fuzzing at scale to find tens of thousands of security vulnerabilities and stability bugs. This summer, as part of Google’s OSS internship initiative, we hosted 50 interns to improve the state of fuzz testing in the open source ecosystem.
The fuzzing interns worked towards integrating new projects and improving existing ones in OSS-Fuzz, our continuous fuzzing service for the open source community (which has 350+ projects, 22,700 bugs, 89% fixed). Several widely used open source libraries including but not limited to nginx, postgresql, usrsctp, and openexr, now have continuous fuzzing coverage as a result of these efforts.
Another group of interns focused on improving the security of the Linux kernel. syzkaller, a kernel fuzzing tool from Google, has been instrumental in finding kernel vulnerabilities in various operating systems. The interns were tasked with improving the fuzzing coverage by adding new descriptions to syzkaller like ip tunnels, io_uring, and bpf_lsm for example, refining the interface description language, and advancing kernel fault injection capabilities.
Some interns chose to write fuzzers for Android and Chrome, which are open source projects that billions of internet users rely on. For Android, the interns contributed several new fuzzers for uncovered areas – network protocols such as pppd and dns, audio codecs like monoblend, g722, and android framework. On the Chrome side, interns improved existing blackbox fuzzers, particularly in the areas: DOM, IPC, media, extensions, and added new libprotobuf-based fuzzers for Mojo.
Our last set of interns researched quite a few under-explored areas of fuzzing, some of which were fuzzer benchmarking, ML based fuzzing, differential fuzzing, bazel rules for build simplification and made useful contributions.
QUIC is a new networking transport protocol that combines the features of TCP, TLS, and more. HTTP/3 is the latest version of HTTP, the protocol that carries the vast majority of Web traffic. HTTP/3 only runs over QUIC.
QUIC was initially developed by Google and first announced in 2013. Since then, the protocol has matured, and is now responsible for carrying over a third of Google traffic. In 2015, Google brought QUIC to the IETF (the standards organization responsible for maintaining the Internet’s protocols) and the IETF has been improving QUIC by making many changes to it. At this point, there are now two similar but different protocols: Google QUIC and IETF QUIC. The QUIC team at Google has been involved in the IETF process from the start, but we’ve been using Google QUIC in Chrome while working on implementing IETF QUIC. We’ve put tremendous effort into evolving Google QUIC over the last five years to track changes at IETF, and the current latest Google QUIC version (Q050) has many similarities with IETF QUIC. But up until now, the majority of Chrome users didn’t communicate with IETF QUIC servers without enabling some command-line options.
Today this changes. We’ve found that IETF QUIC significantly outperforms HTTP over TLS 1.3 over TCP. In particular, Google search latency decreases by over 2%. YouTube rebuffer time decreased by over 9%, while client throughput increased by over 3% on desktop and over 7% on mobile. We’re happy to announce that Chrome is rolling out support for IETF QUIC (specifically, draft version h3-29). Today 25% of Chrome Stable users are using h3-29, and we plan on increasing that number over the coming weeks as we continue to monitor performance data. Chrome will actively support both IETF QUIC h3-29 and Google QUIC Q050 to provide servers that support Q050 with time to update to IETF QUIC.
Chrome m85 doesn’t yet support IETF QUIC 0-RTT, so we expect these performance numbers to look even better once we launch 0-RTT support for IETF QUIC in the coming months.
Since the subsequent IETF drafts 30 and 31 do not have compatibility-breaking changes, we currently are not planning to change the over-the-wire identifier. What this means is that while we’ll keep tracking changes in the IETF specification, we will be deploying them under the h3-29/0xff00001d name. We therefore recommend that servers keep support for h3-29 until the final RFCs are complete if they wish to interoperate with Chrome. However, if the IETF were to make compatibility-breaking changes in a future draft, Chrome will revisit this decision.
The authors would like to thank the entire QUIC team at Google for all their hard work leading up to this announcement. We’re incredibly proud of what we’ve achieved together. We would also like to thank everyone who has contributed to QUIC at the IETF, and all of the former members of the QUIC team at Google, without whom none of this would have been possible.
Posted by
Last year we announced a new initiative (known as Privacy Sandbox) to develop a set of open standards to fundamentally enhance privacy on the web. With Privacy Sandbox we’ve been exploring privacy-preserving mechanisms with the web community that protect user data and prevent intrusive cross-site tracking. Our aim is to preserve the vitality of the open web by continuing to enable the rich, quality content and services that people expect, but with even stronger guarantees of privacy and safety. Today we’re sharing progress on this long-term initiative and asking for your continued help in increasing the privacy of web browsing.
In January we shared our intent to develop privacy-preserving open-standards that will render third-party cookies obsolete. Since then, Google and others have proposed several new APIs to address use cases like fraud protection, ad selection, and conversion measurement without allowing users’ activity to be tracked across websites. Following web community input, some of these solutions are now available for experimental testing via Chrome origin trials:
- Click Conversion Measurement API opened up for testing in September and aims to enable marketers to know whether an ad click resulted in a conversion (for example, a purchase or a sign-up) on another site, without connecting the identity of the user across both sites.
- Trust Tokens opened up for testing in July and is intended to support a number of use cases evaluating a user’s authenticity, including combating fraud.
If you integrate APIs into your products and services, you can register for access to these and other APIs through Chrome origin trials. We encourage ecosystem stakeholders to participate and share their feedback and results. Developing and implementing web standards which change the core architecture of the web is a complex process, so we are taking a long-term, collaborative approach.
We’re also continuing our work to make current web technologies more secure and private.
- Earlier this year Chrome started limiting cross-site tracking by treating cookies that don’t include a SameSite label as first-party only, and requiring cookies to be labeled and accessed over HTTPS in order to be available in third-party contexts. With this update — which Edge and Firefox are in the process of adopting — third-party cookies are no longer sent for the 99.9% of registered domains that do not require them, improving privacy and security for the vast majority of sites on the web.
- In a release early next year, Chrome will also strengthen protection against additional types of network attacks that could hijack the users’ privileged credentials to perform malicious actions on their accounts.
We’re also rolling out changes in Chrome to mitigate deceptive and intrusive tracking techniques, such as fingerprinting.
- In September we rolled out an update to prevent inadvertent sharing of information such as users’ names and access tokens. When users navigate from one site to another we are reducing the information from the originating page’s URL that is sent to the destination site by default.
- Also in September, we extended support of Secure DNS in Chrome beyond desktop to Android. Secure DNS is designed to improve user safety and privacy while browsing the web by automatically switching to DNS-over-HTTPS if the user’s current provider supports it.
- Coming soon, we’re also closing the ability for a site to observe other sites that a user might have visited through caching mechanisms.
As always, we encourage you to give feedback on the web standards community proposals via GitHub and make sure they address your needs. And if they don’t, file issues through GitHub or email the W3C group. If you rely on the web for your business, please ensure your technology vendors engage in this process and that the trade groups who represent your interests are actively engaged.
We are appreciative of the continued engagement as we build a more trustworthy and sustainable web together. We will continue to keep everyone posted on the progress of efforts to increase the privacy of web browsing.
We are very pleased to announce the latest Google Open Source Peer Bonus winners!
The Google Open Source Peer Bonus program rewards external open source contributors nominated by Googlers for their exceptional contributions to open source. Historically, the program was primarily focused on rewarding developers. Over the years the program has evolved—rewarding not just software engineers contributors from every part of open source—including technical writers, user experience and graphic designers, community managers and marketers, mentors and educators, ops and security experts.
Although the majority of recipients in this round were recognized for their code contributions, more than 40% of the successful nominations included tooling work, community work, and documentation. (Some contributors were recognized for their work in more than one area.)
Below is the list of current winners who gave us permission to thank them publicly:
| Winner | Project |
| Xihan Li | A Concise Handbook of TensorFlow 2 |
| Alain Schlesser | AMP Plugin for WordPress |
| Pierre Gordon | AMP Plugin for WordPress |
| Catherine Houle | AMP Project |
| Quyen Le Hoang | ANGLE |
| Kamil Bregula | Apache Airflow |
| László Kiss Kollár | auditwheel/manylinux |
| Jack Neus | Chrome OS Release Branching tool |
| Fabian Henneke | chromium |
| Matt Godbolt | Compiler Explorer |
| Sumeet Pawnikar | coreboot |
| Hal Seki | covid19 |
| Derek Parker | Delve |
| Alessandro Arzilli | Delve |
| Matthias Sohn | Eclipse Foundation |
| Luca Milanesio | Eclipse Foundation |
| João Távora | eglot |
| Brad Cowie | faucetsdn |
| Harri Hohteri | Firebase |
| Rosário Pereira Fernandes | Firebase |
| Peter Steinberger | Firebase iOS, CocoaPods |
| Eduardo Silva | Fluent Bit |
| Matthias Sohn | Gerrit Code Review |
| Marco Miller | Gerrit Code Review |
| Camilla Löwy | GLFW |
| Akim Demaille | GNU Bison |
| Josh Bleecher Snyder | Go |
| Alex Brainman | Go |
| Richard Musiol | Go |
| Roger Peppe | Go, CUE, gohack |
| Daniel Martí | Go, CUE, many individual repo. |
| Juan Linietsky | Godot Engine |
| Maddy Myers | Google Research Open-COVID-19-Data |
| Pontus Leitzler | govim, gopls |
| Paul Jolly | govim, gopls |
| Parul Raheja | Ground |
| Pau Freixes | gRPC |
| Marius Brehler | IREE |
| George Nachman | iterm2 |
| Kenji Urushima | jsrsasign |
| Jacques Chester | KNative |
| Markus Thömmes | Knative Serving |
| Savitha Raghunathan | Kubernetes |
| David Anderson | libdwarf |
| Florian Westphal | Linux kernel |
| Jonas Bernoulli | magit |
| Hugo van Kemenade | Many open-source Python projects |
| Jeff Lockhart | Maps SDK for Android Utility Library |
| Claude Vervoort | Moodle |
| Jared McNeill | NetBSD |
| Nao Yonashiro | nginx-sxg-module |
| Geoffrey Booth | Node.js |
| Gus Caplan | Node.js |
| Guy Bedford | Node.js |
| Samson Goddy | Open Source Community Africa |
| Daniel Dyla | OpenTelemetry |
| Leighton Chen | OpenTelemetry |
| Shivkanya Andhare | OpenTelemetry |
| Bartlomiej Obecny | OpenTelemetry |
| Philipp Wagner | OpenTitan, Ibex, CocoTB |
| Srijan Reddy | Oppia |
| Chris S | Oppia |
| Bastien Guerry | Org mode |
| Gary Kramlich | Pidgin Lead Developer |
| Hassan Kibirige | plotnine |
| Abigail Dogbe | PyLadies Ghana |
| David Hewitt | PyO3 |
| Yuji Kanagawa | PyO3 |
| Mannie Young | Python Ghana |
| Alex Bradbury | RISC-V LLVM, Ibex, OpenTitan |
| Lukas Taegert-Atkinson | Rollup.js |
| Sanil Raut | Shaka Packager |
| Richard Hallows | stylelint |
| Luke Edwards | Svelte and Node Libraries |
| Zoe Carver | Swift Programming Language |
| Nick Lockwood | SwiftFormat |
| Priti Desai | Tekton |
| Sayak Paul | TensorFlow |
| Lukas Geiger | TensorFlow |
| Margaret Maynard-Reid | TensorFlow |
| Gabriel de Marmiesse | TensorFlow Addons |
| Jared Morgan | The Good Docs Project |
| Jo Cook | The Good Docs Project, GeoNetwork, Portable GIS, Various Open Source Geospatial Foundation communities |
| Ricky Mulyawan Suryadi | Tink JNI Examples |
| Nicholas Marriott | Tmux |
| Michael Tüxen | usrsctp |
| Seth Brenith | V8 |
| Ramya Rao | VS Code Go |
| Philipp Hancke | WebRTC |
| Jason Donenfeld | WireGuard |
By Maria Tabak and Erin McKean, Program Managers – Google Open Source Programs Office
The Kubernetes Ingress API, first introduced in late 2015 as an experimental beta feature, has finally graduated as a stable API and is included in the recent 1.19 release of Kubernetes.
The goal of the Ingress API is to provide a simple uniform means of describing the routing of HTTP or HTTPS traffic from outside a cluster to backend services within a cluster; independent of the Ingress Controller being used. An Ingress controller is a 3rd party application, such as Nginx or an external service like the Google Cloud Load Balancer (GCLB), that performs the actual routing of the HTTP(S) traffic. This uniform API, supported by the Ingress Controllers made it easy to create simple HTTP(S) load balancers, however most use-cases required something more complex.
By early 2019, the Ingress API had remained in beta for close to four years. Beta APIs are not intended to be relied upon for business-critical production use, yet many users were using the Ingress API in some level of production capacity. After much discussion, the Kubernetes Networking Special Interest Group (SIG) proposed a path forward to bring the Ingress API to GA primarily by introducing two changes in Kubernetes 1.18. These were: a new field, pathType, to the Ingress API; and a new Ingress resource type, IngressClass. Combined, they provide a means of guaranteeing a base level of compatibility between different path prefix matching implementations, along with opening the door to further extension by the Ingress Controller developers in a uniform and consistent pattern.
What does this mean for you? You can be assured that the path prefixes you use will be evaluated the same way across Ingress Controllers implementations, and the Ingress configuration sprawl across Annotations, ConfigMaps and CustomResourceDefinitions (CRDs) will be consolidated into a single IngressClass resource type.
pathType
The pathType field specifies one of three ways that an Ingress Object’s path should be interpreted:
- ImplementationSpecific: Path prefix matching is delegated to the Ingress Controller (IngressClass).
- Exact: Matches the URL path exactly (case sensitive)
- Prefix: Matches based on a URL path prefix split by /. Matching is case sensitive and done on a path element by element basis.
NOTE: ImplementationSpecific was configured as the default pathType in the 1.18 release. In 1.19 the defaulting behavior was removed and it MUST be specified. Paths that do not include an explicit pathType will fail validation.
|
Pre Kubernetes 1.18 |
Kubernetes 1.19+ |
|
apiVersion: networking.k8s.io/v1beta1 |
apiVersion: networking.k8s.io/v1 |
These changes not only make room for backwards-compatible configurations with the ImplementationSpecific pathType, but also enables more portable workloads between Ingress Controllers with Exact or Prefix pathType.
IngressClass
The new IngressClass resource takes the place of various different Annotations, ConfigMaps, Environment Variables or Command Line Parameters that you would regularly pass to an Ingress Controller directly. Instead, it has a generic parameters field that can be used to reference controller specific configuration.
Example IngressClass Resource
|
apiVersion: networking.k8s.io/v1 |
Source: https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-class
In this example, the parameters resource would include configuration options implemented by the example.com/ingress-controller ingress controller. These items would not need to be passed as Annotations or a ConfigMap as they would in versions prior to Kubernetes 1.18.
How do you use IngressClass with an Ingress Object? You may have caught it in the earlier example, but the Ingress resource’s spec has been updated to include an ingressClassName field. This field is similar to the previous kubernetes.io/ingress.class annotation but refers to the name of the corresponding IngressClass resource.
Other Changes
Several other small changes went into effect with the graduation of Ingress to GA in 1.19. A few fields have been remapped/renamed and support for resource backends has been added.
Remapped Ingress Fields
- v1beta1: spec.backend -> v1: spec.defaultBackend
- v1beta1: backend.service.name -> v1: backend.service.name
- v1beta1: backend.servicvePort (servicePort is now multiple fields)
Resource Backend
A Resource Backend is essentially a pointer or ObjectRef (apiGroup, kind, name) to another resource in the same namespace. Why would you want to do this? Well, it opens the door to all sorts of future possibilities such as routing to static object storage hosted in GCS or S3, or another internal form of storage.
NOTE: Resource Backend and Service Backends are mutually exclusive. Only one field can be specified at a time.
Deprecation Notice
Should you migrate right now (September 2020)? Not yet. The majority of Ingress Controllers have not added support for the new GA Ingress API. Ingress-GCE, the Ingress Controller for Google Kubernetes Engine (GKE) should be updated to support the Ingress GA API in Q4 2020. Keep your eyes on the GKE rapid release channel to stay up to date on it, and Kubernetes 1.19’s availability.
What’s Next for Ingress?
The Ingress API has had a rough road getting to GA. It is an essential resource for many, and the changes that have been introduced help manage that complexity while keeping it relatively light-weight. However, even with the added flexibility that has been introduced it doesn’t cover a variety of complex use-cases.
SIG Network has been working on a new API referred to as “Service APIs” that takes into account the lessons learned from the previous efforts of working on Ingress. These Service APIs are not intended to replace Ingress, but instead compliment it by providing several new resources that could enable more complex workflows.
By Bob Killen, Program Manager, Google Open Source Programs Office
Today we’re announcing that HarbourBridge—an open source toolkit that automates much of the manual work of evaluating and assessing Cloud Spanner—supports migrations from MySQL, in addition to existing support for PostgreSQL. This provides a zero-configuration path for MySQL users to try out Cloud Spanner. HarbourBridge bootstraps early stages of migration, and helps get you to the meaty issues as quickly as possible.
Core capabilities
At its core, HarbourBridge provides an automated workflow for loading the contents of an existing MySQL or PostgreSQL database into Spanner. It requires zero configuration—no manifests or data maps to write. Instead, it imports the source database, builds a Spanner schema, creates a new Spanner database populated with data from the source database, and generates a detailed assessment report. HarbourBridge can either import dump files (from mysqldump or pg_dump) or directly connect to the source database. It is intended for loading databases up to a few tens of GB for evaluation purposes, not full-scale migrations.
Bootstrap early-stage migration
HarbourBridge bootstraps early-stage migration to Spanner by using an existing MySQL or PostgreSQL source database to quickly get you running on Spanner. It generates an assessment report with an overall migration-fitness score for Spanner, a table-by-table analysis of type mappings and a list of features used in the source database that aren’t supported by Spanner.
View HarbourBridge as a way to get up and running quickly, so you can focus on critical things like tuning performance and getting the most out of Spanner. You will need to tweak and enhance what HarbourBridge produces—more on that later.
Getting started
HarbourBridge can be used with the Cloud Spanner Emulator, or directly with a Cloud Spanner instance. The Emulator is a local, in-memory emulation of Spanner that implements the same APIs as Cloud Spanner’s production service, and allows you to try out Spanner’s functionality without creating a GCP Project. The HarbourBridge README contains a step-by-step quick-start guide for using the tool with a Cloud Spanner instance.
Together, HarbourBridge and the Cloud Spanner Emulator provide a lightweight, open source toolchain to experiment with Cloud Spanner. Moreover, when you want to proceed to performance testing and tuning, switching to a production Cloud Spanner instance is a simple configuration change.
To get started on using HarbourBridge with the Emulator, follow the Emulator instructions. In particular, start the Emulator using Docker and configure the SPANNER_EMULATOR_HOST environment variable (this tells the Cloud Spanner Client libraries to use the Emulator).
Next, install Go and configure the GOPATH environment variable if they are not already part of your environment. Now you can download and install HarbourBridge using
GO111MODULE=on
go get github.com/cloudspannerecosystem/harbourbridge
It should be installed as $GOPATH/bin/harbourbridge. To use HarbourBridge on a MySQL database, run mysqldump and pipe its output to HarbourBridge
mysqldump <opts> db | $GOPATH/bin/harbourbridge -driver=mysqldump
Similarly, to use HarbourBridge on a PostgreSQL database, run
pg_dump <opts> db | $GOPATH/bin/harbourbridge -driver=pg_dump
Sample dump files
If you don’t have ready access to a MySQL or PostgreSQL database, the HarbourBridge github repository has some samples. The files cart.mysqldump and cart.pg_dump contain mysqldump and pg_dump output for a very basic shopping cart application (just two tables, one for products and one for user carts). The files singers.mysqldump and singers.pg_dump contain mysqldump and pg_dump output for a version of the Cloud Spanner singers example. To use HarbourBridge on cart.mysqldump, download the file locally and run
$GOPATH/bin/harbourbridge -driver=mysqldump < cart.mysqldump
Next steps
The schema created by HarbourBridge provides a starting point for evaluation of Spanner. While it preserves much of the core structure of your MySQL or PostgreSQL schema, data types will be mapped based on the types supported by Spanner, and unsupported features will be dropped e.g. functions, sequences, procedures, triggers and views. See the assessment report as well as HarbourBridge’s Schema conversion documentation for details.
To test Spanner’s performance, you will need to switch from the Emulator to a Cloud Spanner instance. The HarbourBridge quick-start guide provides details of how to set up a Cloud Spanner instance. To have HarbourBridge use your Cloud Spanner instance instead of the Emulator, simply unset the SPANNER_EMULATOR_HOST environment variable (see the Emulator documentation for context).
To optimize your Spanner performance, carefully review choices of primary keys and indexes—see Keys and indexes. Note that HarbourBridge preserves primary keys from the source database but drops all other indexes. This means that the out-of-the-box performance you get from the schema created by HarbourBridge can be significantly impacted. If this is the case, add appropriate Secondary indexes. In addition, consider using Interleaved tables to optimize table layout and improve the performance of joins.
Recap
HarbourBridge is an open source toolkit for evaluating and assessing Cloud Spanner using an existing MySQL or PostgreSQL database. It automates many of the manual steps so that you can quickly get to important design, evaluation and performance issues, such as. refining choice of primary keys, tuning of indexes, and other optimizations.
We encourage you to try out HarbourBridge, send feedback, file issues, fork and modify the codebase, and send PRs for fixes and new functionality. We have big plans for HarbourBridge, including the addition of user-guided schema conversion (to customize type mappings and provide a guided exploration of indexing, primary key choices, and use of interleaved tables), as well as support for more databases. HarbourBridge is part of the Cloud Spanner Ecosystem, owned and maintained by the Cloud Spanner user community. It is not officially supported by Google as part of Cloud Spanner.
By Nevin Heintze, Cloud Spanner
Science Journal is an open source mobile app that enables students in K-12 classrooms to conduct fun, hands-on science experiments using smart devices. Since its launch in May 2016, Science Journal has gone through quite a journey, from collaborating with rock stars to supporting classrooms, through an integration with Google Drive. Now we’re pleased to share that Science Journal has graduated from Google and moved over to Arduino, the makers of the popular open source Arduino microcontrollers for students, hobbyists, and professionals around the world. Arduino Science Journal is free, open sourced, and available for download today on Android and iOS.
We’re thrilled to be handing the project to a partner that has a long history of supporting open source and education. The Arduino Science Kit for middle school students was developed in 2019 in partnership with Science Journal. The Arduino Science Journal Android source code and iOS source code, are already available on GitHub along with the Science Journal Arduino firmware. We’ve put a lot of time and energy into making Science Journal a great app for students and science enthusiasts everywhere, and we’re confident that it will continue to thrive in its new home.
Although Google’s Science Journal apps are still available on the App and Play Store today, these apps will no longer be supported after December 11, 2020, at which point Google Drive Syncing will stop working and Google’s versions of the apps will no longer be available for download. However, existing Science Journal experiments can be exported from Google’s Science Journal apps and imported into Arduino Science Journal at any time. You can find more information about this handoff in our Help Center article.
We see this change as a win for Google, Arduino, Science Journal, and for open source overall. Since Science Journal is an app for kids and schools, we wanted to be particularly careful with this transition. By supporting Arduino in releasing their own version of Science Journal and forking our code on GitHub, we were able to effectively hand off the project without transferring any user data or Intellectual Property. We hope this approach can serve as an effective model for future projects that need to reallocate their resources but don’t want to let down their users (as we like to say: focus on the user, and all else will follow).
Moving forward, all future updates will be happening through Arduino’s versions of the apps. You can stay up-to-date on the Arduino Science Journal website and experiment with their new hands-on activities, and if you have any questions, you can contact them on the Arduino Science Journal Forum.
Although the Science Journal project is moving on from Google, we still think data and scientific literacy are critically important for present and future generations, now more than ever. With the ubiquity of smart devices in classrooms and at home, we think Science Journal remains the perfect solution for parents and teachers looking to provide students with hands-on learning opportunities during this time period. We hope you enjoy using Science Journal as much as we have, and we’re excited to see how the project will continue to evolve moving forward.
For many, gazing at an old photo of a city can evoke feelings of both nostalgia and wonder. We have Google Street View for places in the present day, but what about places in the past? What was it like to walk through Manhattan in the 1940s? To create a rewarding “time travel” experience for both research and entertainment purposes, Google Research is launching Kartta Labs, an open source, scalable system on Google Cloud and Kubernetes that tackles the difficult problem of reconstructing what cities looked like in the past from scarce historical maps and photos.
Kartta Labs consists of three main parts:
- A temporal map server, which shows how maps change over time;
- A crowdsourcing platform, which allows users to upload historical maps of cities, georectify, and vectorize them (i.e. match them to real world coordinates);
- And an upcoming 3D experience platform, which runs on top of maps creating the 3D experience by using deep learning to reconstruct buildings in 3D from limited historical images and maps data.
Maps & Crowdsourcing
Kartta Labs is a growing suite of open source tools that work together to create a map server with a time dimension, allowing users to populate the service with historically accurate data.
Warper
The entry point to crowdsourcing is Warper, an open source web app based on MapWarper that allows users to upload historical images of maps and georectify them by finding control points on the historical map and corresponding points on a base map.
Once a user uploads a scanned historical map, Warper makes a best guess of the map’s geolocation by extracting textual information from the map. This initial guess is used to place the map roughly in its location and allow the user to georeference the map pixels by placing pairs of control points on the historical map and a reference map. Given the georeferenced points, the application warps the image such that it aligns well with the reference map.
Warper runs as a Ruby on Rails application using a number of open source geospatial libraries and technologies, including but not limited to PostGIS and GDAL. The resulting maps can be exported in PNG, GeoTIFF, and other open formats. Warper also runs a raster tiles server that serves each georectified map at a tile URL. This raster tile server is used to load the georectified map as a background in the Editor application that is described next.
Editor
Editor is an open source web application which is a customized version of the OpenStreetMap editor; customizations include support for time dimension and integration with the other tools in the Kartta Labs suite. Editor allows users to load the georectified historical maps and trace their geographic features (e.g., building footprints, roads, etc.). This traced data is stored in vector format.
Extracted geometries in vector format, as well as metadata (e.g., address, name, and start or end dates), are stored in a geospatial database that can be queried, edited, styled, and rendered into new maps.
Kartta
Finally, the temporal map front end, Kartta (based on Tegola), visualizes the vector tiles allowing the users to navigate historical maps in space and time. Kartta works like any familiar map application (such as Google Maps), but also has a time slider so the user can choose the year at which they want to see the map. By moving the time slider, the user is able to see how features in the map, such as buildings and roads, changes over time.
3D Experience
To actually create the “time traveling” 3D experience, the forthcoming 3D Models module aims to reconstruct the detailed full 3D structures of historical buildings. The module will associate images with maps data, organize these 3D models properly in one repository, and render them on the historical maps with a time dimension.
Preliminary Results
 |
| Figure 2 – Bird’s eye view of 3D-reconstructed Chelsea, Manhattan with a time slider |
 |
| Figure 3 – Street level view of 3D-reconstructed Chelsea, Manhattan |
Conclusion
Chrome on iOS sends two different User-Agent strings, depending on the version of the site being requested by the user.
In M84 and earlier, the User-Agent string sent when the Request Desktop Site option was selected matched the string used by Safari Desktop.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1.1 Safari/605.1.15
Starting with M85, the User-Agent string sent with the Request Desktop Site option changes to include the CriOS/<MajorVersion> tag.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/85 Version/11.1.1 Safari/605.1.15
Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e Safari/602.1
The goal of this change is to allow developers to tailor the user experiences to account for differences between Chrome and Safari on iOS. This change adds more information to the Desktop User-Agent string, but including the browser name and major version in the User Agent header is still in line with the goals of Chrome’s User Agent information reduction plans.
English
Arabic
Chinese (Simplified)
Chinese (Traditional)
French
Italian
Japanese
Korean
Portuguese
Spanish
Swedish