Firefox 68 landed earlier this month with a bunch of CSS additions and changes. In this blog post we will take a look at some of the things you can expect to find, that might have been missed in earlier announcements.
CSS Scroll Snapping
The headline CSS feature this time round is CSS Scroll Snapping. I won’t spend much time on it here as you can read the blog post for more details. The update in Firefox 68 brings the Firefox implementation in line with Scroll Snap as implemented in Chrome and Safari. In addition, it removes the old properties which were part of the earlier Scroll Snap Points Specification.
The ::marker pseudo-element
The ::marker pseudo-element lets you select the marker box of a list item. This will typically contain the list bullet, or a number. If you have ever used an image as a list bullet, or wrapped the text of a list item in a span in order to have different bullet and text colors, this pseudo-element is for you!
With the marker pseudo-element, you can target the bullet itself. The following code will turn the bullet on unordered lists to hot pink, and make the number on an ordered list item larger and blue.
ul ::marker {
color: hotpink;
}
ol ::marker {
color: blue;
font-size: 200%;
}
With ::marker we can style our list markers
There are only a few CSS properties that may be used on ::marker. These include all font properties. Therefore you can change the font-size or family to be something different to the text. You can also color the bullets as shown above, and insert generated content.
Using ::marker on non-list items
A marker can only be shown on list items, however you can turn any element into a list-item by using display: list-item. In the example below I use ::marker, along with generated content and a CSS counter. This code outputs the step number before each h2 heading in my page, preceded by the word “step”. You can see the full example on CodePen.
h2 {
display: list-item;
counter-increment: h2-counter;
}
h2::marker {
content: "Step: " counter(h2-counter) ". ";
}If you take a look at the bug for the implementation of ::marker you will discover that it is 16 years old! You might wonder why a browser has 16 year old implementation bugs and feature requests sitting around. To find out more read through the issue, where you can discover that it wasn’t clear originally if the ::marker pseudo-element would make it into the spec.
There were some Mozilla-specific properties that achieved the result developers were looking for with something like ::marker. The properties ::moz-list-bullet and ::moz-list-marker allowed for the styling of bullets and markers respectively, using a moz- vendor prefix.
The ::marker pseudo-element is standardized in CSS Lists Level 3, and CSS Pseudo-elements Level 4, and currently implemented as of Firefox 68, and Safari. Chrome has yet to implement ::marker. However, in most cases you should be able to use ::marker as an enhancement for those browsers which support it. You can allow the markers to fall back to the same color and size as the rest of the list text where it is not available.
CSS Fixes
It makes web developers sad when we run into a feature which is supported but works differently in different browsers. These interoperability issues are often caused by the sheer age of the web platform. In fact, some things were never fully specified in terms of how they should work. Many changes to our CSS specifications are made due to these interoperability issues. Developers depend on the browsers to update their implementations to match the clarified spec.
Most browser releases contain fixes for these issues, making the web platform incrementally better as there are fewer issues for you to run into when working with CSS. The latest Firefox release is no different – we’ve got
At Mozilla, we are continuing to experiment with DNS-over-HTTPS (DoH), a new network protocol that encrypts Domain Name System (DNS) requests and responses. This post outlines a new study we will be conducting to gauge how many Firefox users in the United States are using parental controls or enterprise DNS configurations.
With previous studies, we have tried to understand the performance impacts of DoH, and the results have been very promising. We found that DoH queries are typically the same speed or slightly slower than DNS queries, and in some cases can be significantly faster. Furthermore, we found that web pages that are hosted by Akamai–a content distribution network, or “CDN”–have similar performance when DoH is enabled. As such, DoH has the potential to improve user privacy on the internet without impeding user experience.
Now that we’re satisfied with the performance of DoH, we are shifting our attention to how we will interact with existing DNS configurations that users have chosen. For example, network operators often want to filter out various kinds of content. Parents and schools in particular may use “parental controls”, which block access to websites that are considered unsuitable for children. These controls may also block access to malware and phishing websites. DNS is commonly used to implement this kind of content filtering.
Similarly, some enterprises set up their own DNS resolvers that behave in special ways. For example, these resolvers may return a different IP address for a domain name depending on whether the user that initiated the request is on a corporate network or a public network. This behavior is known as “split-horizon”, and it is often to host a production and development version of a website. Enabling DoH in this scenario could unintentionally prevent access to internal enterprise websites when using Firefox.
We want to understand how often users of Firefox are subject to these network configurations. To do that, we are performing a study within Firefox for United States-based users to collect metrics that will help answer this question. These metrics are based on common approaches to implementing filters and enterprise DNS resolvers.
Detecting DNS-based parental controls
This study will generate DNS lookups from participants’ browsers to detect DNS-based parental controls. First, we will resolve test domains operated by popular parental control providers to determine if parental controls are enabled on a network. For example, OpenDNS operates exampleadultsite.com. It is not actually an adult website, but it is present on the blocklists for several parental control providers. These providers often block access to such websites by returning an incorrect IP address for DNS lookups.
As part of this test, we will resolve exampleadultsite.com. According to OpenDNS, this domain name should only resolve to the address 146.112.255.155. Thus, if a different address is returned, we will infer that DNS-based parental controls have been configured. The browser will not connect to, or download any content from the website.
We will also attempt to detect when a network has forced “safe search” versions of Google and YouTube for its users. The way that safe search works is that the network administrator configures their resolver to redirect DNS requests for a search provider to a “safe” version of the website. For example, a network administrator may force all users that look up www.google.com to instead look up forcesafesearch.google.com. When the browser connects to the IP address for forcesafesearch.google.com, the search provider knows that safe search is enabled and returns filtered search results.
We will resolve the unrestricted domain names provided by Google and YouTube from the addon, and then resolve the safe search domain names. Importantly, the safe search domain names for Google and YouTube are hosted on fixed IP addresses. Thus, if the IP address for an unrestricted and safe search domain name match, we will infer that parental controls are enabled. The tables below show the domain names we will resolve to detect safe search.
Hello and welcome to another issue of This Week in Rust! Rust is a systems language pursuing the trifecta: safety, concurrency, and speed. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.
This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.
Updates from Rust Community
News & Blog Posts
- The Rust compiler is still getting faster.
- Unsafe as a human-assisted type system.
- Why does the Rust compiler not optimize code assuming that two mutable references cannot alias?
- Python vs Rust for neural networks.
- Dependency management and trust scaling.
Crate of the Week
This week's crate is async-trait, a procedural macro to allow async fns in trait methods.
Thanks to Ehsan M. Kermani for the suggestion!
Submit your suggestions and votes for next week!
Call for Participation
Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!
Some of these tasks may also have mentors available, visit the task page for more information.
If you are a Rust project owner and are looking for contributors, please submit tasks here.
Updates from Rust Core
324 pull requests were merged in the last week
- Add support for UWP targets
- Add
riscv32i-unknown-none-elftarget - Update wasm32 support for LLVM 9
- Move unescape module to rustc_lexer
- Make the parser TokenStream more resilient after mismatched delimiter recovery
- Improve diagnostics for _ const/static declarations
- Avoid ICE when referencing desugared local binding in borrow error
- Suggest trait bound on type parameter when it is unconstrained
- Allow lifetime elision in
Pin<&(mut) Self> - Stop bare trait lint applying to macro call sites
- Add note suggesting to borrow a String argument to find
- Add method disambiguation help for trait implementation
I often need to quickly test a snippet of HTML, mostly to see how it interacts with our accessibility APIs.
Instead of creating some throwaway HTML file each time, I find it easier to paste in the HTML in devtools, or even make a data URI.
Last week I spent an hour creating an extension that allows you to just paste some HTML into the address bar and have it rendered immediately.
You just need to prefix it with the html keyword, and you’re good to go. Like this html
Hello, World!
.You can download it from github.
There might be other extensions or ways of doing this, but it was a quick little project.
https://blog.monotonous.org/2019/07/30/html-snippet-extension/
There's a movement building within tech. Workers are demanding higher standards from their companies — and because of their unique skills and talent, they have the leverage to get attention. Walkouts and sit-ins. Picket protests and petitions. Shareholder resolutions, and open letters. These are the new tools of tech workers, increasingly emboldened to speak out. And, as they do that, they expose the underbellies of their companies' ethics and values or perceived lack of them.
In this episode of IRL, host Manoush Zomorodi meets with Rebecca Stack-Martinez, an Uber driver fed up with being treated like an extension of the app; Jack Poulson, who left Google over ethical concerns with a secret search engine being built for China; and Rebecca Sheppard, who works at Amazon and pushes for innovation on climate change from within. EFF Executive Director Cindy Cohn explains why this movement is happening now, and why it matters for all of us.
IRL is an original podcast from Firefox. For more on the series go to irlpodcast.org
Rebecca Stack-Martinez is a committee member for Gig Workers Rising.
Here is Jack Poulson's resignation letter to Google. For more, read Google employees' open letter against Project Dragonfly.
Check out Amazon employees' open letter to Jeff Bezos and Board of Directors asking for a better plan to address climate change.
Cindy Cohn is the Executive Director of the Electronic Frontier Foundation. EFF is a nonprofit that defends civil liberties in the digital world. They champion user privacy, free expression, and innovation through impact litigation, policy analysis, grassroots activism, and technology development.
We are excited to tell you about our experimental Mixed Reality editor, an experiment we did in the spring to explore online editing in MR stories. What’s that? You haven’t heard of MrEd? Well please allow us to explain.
For the past several months Blair, Anselm and I have been working on a visual editor for WebXR called the Mixed Reality Editor, or MrEd. We started with this simple premise: non-programmers should be able to create interactive stories and experiences in Mixed Reality without having to embrace the complexity of game engines and other general purpose tools. We are not the first people to tackle this challenge; from visual programming tools to simplified authoring environments, researchers and hobbyists have grappled with this problem for decades.
Looking beyond Mixed Reality, there have been notable successes in other media. In the late 1980s Apple created a ground breaking tool for the Macintosh called Hypercard. It let people visually build applications at a time when programming the Mac required Pascal or assembly. It did this by using the concrete metaphor of a stack of cards. Anything could be turned into a button that would jump the user to another card. Within this simple framework people were able to create eBooks, simple games, art, and other interactive applications. Hypercard’s reliance on declaring possibly large numbers of “visual moments” (cards) and using simple “programming” to move between them is one of the inspirations for MrEd.
We also took inspiration from Twine, a web-based tool for building interactive hypertext novels. In Twine, each moment in the story (seen on the screen) is defined as a passage in the editor as a mix of HTML content and very simple programming expressions executed when a passage is displayed, or when the reader follows a link. Like Hypercard, the author directly builds what the user sees, annotating it with small bits of code to manage the state of the story.
No matter what the medium — text, pictures, film, or MR — people want to tell stories. Mixed Reality needs tools to let people easily tell stories by focusing on the story, not by writing a simulation. It needs content focused tools for authors, not programmers. This is what MrEd tries to be.
Scenes Linked Together
At first glance, MrEd looks a lot like other 3D editors, such as Unity3D or Amazon Sumerian. There is a scene graph on the left, letting authors create scenes, add anchors and attach content elements under them. Select an item in the graph or in the 3D windows, and a property pane appears on the right. Scripts can be attached to objects. And so on. You can position your objects in absolute space (good for VR) or relative to other objects using anchors. An anchor lets you do something like look for this poster in the real world, then position this text next to it, or look for this GPS location and put this model on it. Anchors aren’t limited to basic can also express more semantically meaningful concepts like find the floor and put this on it (we’ll dig into this in another article).
Dig into the scene graph on the left, and differences appear. Instead of editing a single world or game level, MrEd uses the metaphor of a series of scenes (inspired by Twine’s passages and Hypercard’s cards). All scenes in the project are listed, with each scene defining what you see at any given point: shapes, 3D models, images, 2D text and sounds. You can add interactivity by attaching behaviors to objects for things like ‘click to navigate’ and ‘spin around’. The story advances by moving from scene to scene; code to keep track of story state is typically executed on these scene transitions, like Hypercard and Twine. Where most 3D editors force users to build simulations for their experiences, MrEd lets authors create stores that feel more like “3D flip-books”. Within a scene, the individual elements can be animated, move around, and react to the user (via scripts), but the story progresses by moving from scene to scene. While it is possible to create complex individual scenes that begin to feel like a Unity scene, simple stories can be told through sequences of simple scenes.
We built MrEd on Glitch.com, a free web-based code editing and hosting service. With a little hacking we were able to put an entire IDE and document
Summary / TL;DR (new developments since last meeting in bold)
| Project | What’s in it? | Status |
| C++20 | See below | On track |
| Library Fundamentals TS v3 | See below | Under development |
| Concepts | Constrained templates | In C++20 |
| Parallelism TS v2 | Task blocks, library vector types and algorithms, and more | Published! |
| Executors | Abstraction for where/how code runs in a concurrent context | Targeting C++23 |
| Concurrency TS v2 | See below | Under active development |
| Networking TS | Sockets library based on Boost.ASIO | Published! Not in C++20. |
| Ranges | Range-based algorithms and views | In C++20 |
| Coroutines | Resumable functions (generators, tasks, etc.) | In C++20 |
| Modules | A component system to supersede the textual header file inclusion model | In C++20 |
| Numerics TS | Various numerical facilities | Under active development |
| C++ Ecosystem TR | Guidance for build systems and other tools for dealing with Modules | Under active development |
| Contracts | Preconditions, postconditions, and assertions | Pulled from C++20, now targeting C++23 |
| Pattern matching | A match-like facility for C++ |
Under active development, targeting C++23 |
| Reflection TS | Static code reflection mechanisms | Publication imminent |
| Reflection v2 | A value-based constexpr formulation of the Reflection TS facilities |
Under active development, targeting C++23 |
| Metaclasses | Next-generation reflection facilities | Early development |
A few links in this blog post may not resolve until the committee’s post-meeting mailing is published (expected within a few days of August 5, 2019). If you encounter such a link, please check back in a few days.
Introduction
Last week I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in Cologne, Germany. This was the second committee meeting in 2019; you can find my reports on preceding meetings here (February 2019, Kona) and here (November 2018, San Diego), and previous ones linked from those. These reports, particularly the Kona one, provide useful context for this post.
This week the committee reached a very important milestone in the C++20 publication schedule: we approved the C++20 Committee Draft (CD), a feature-complete draft of the C++20 standard which includes wording for all of the new features we plan to ship in C++20.
The next step procedurally is to send out the C++20 CD to national standards bodies for a formal ISO ballot, where they have the opportunity to comment on it. The ballot period is a few months, and the results will be in by the next meeting, which will be in November in Belfast, Northern Ireland. We will then spend that meeting and the next one addressing the comments, and then publishing a revised draft standard. Importantly, as this is a feature-complete draft, new features cannot be added in response to comments; only bugfixes to existing features can be made, and in rare cases where a serious problem is discovered, a feature can be removed.
Attendance at this meeting once again broke previous records, with over 200 people present for the first time ever. It was observed that one of the likely reasons for the continued upward trend in attendance is the proliferation of domain-specific study groups such as
We are excited to announce that Firefox Reality is now available for the Oculus Quest!
Following our releases for other 6DoF headsets including the HTC Vive Focus Plus and Lenovo Mirage, we are delighted to bring the Firefox Reality VR web browsing experience to Oculus' newest headset.
Whether you’re watching immersive video or meeting up with friends in Mozilla Hubs, Firefox Reality takes advantage of the Oculus Quest’s boost in performance and capabilities to deliver the best VR web browsing experience. Try the new featured content on the FxR home page or build your own to see what you can do in the next generation of standalone virtual reality headsets.
Enhanced Tracking Protection Blocks Sites from Tracking You
To protect our users from the pervasive tracking and collection of personal data by ad networks and tech companies, Firefox Reality has Enhanced Tracking Protection enabled by default. We strongly believe privacy shouldn’t be relegated to optional settings. As an added bonus, these protections work in the background and actually increase the speed of the browser.
Firefox Reality is available in 10 different languages, including Japanese, Korean, Simplified Chinese and Traditional Chinese, with more on the way. You can also use your voice to search the web instead of typing, making it faster and easier to get where you want to go.
Stay tuned in the coming months as we roll out support for the nearly VR-ready WebXR specification, multi-window browsing, bookmarks sync, additional language support and other exciting new features.
Like all Firefox browser products, Firefox Reality is available for free in the Oculus Quest store.
For more information: https://mixedreality.mozilla.org/firefox-reality/
We are excited to announce the launch of the “Mozilla Voice Challenge,” a crowdsourcing competition sponsored by Mozilla and posted on the HeroX platform. The goal of the competition is to better define the voice technology space by creating a “stack” of open source technologies to support the development of new voice-enabled products.
The Power of the Voice
Voice-enabled products are in rapid ascent in both consumer and enterprise markets. The expectations are that in the near future voice interaction will become a key interface for people’s internet-connected lives.
Unfortunately, the current voice product market is heavily dominated by a few giant tech companies. This is unhealthy as it stifles the competition and prevents entry of smaller companies with new and innovative products. Mozilla wants to change that. We want to help opening up the ecosystem. So far there have been two major components in Mozilla’s open source voice tech efforts outside the Firefox browser:
(1) To solve for the lack of available training data for machine-learning algorithms that can power new voice-enabled applications, we launched the Common Voice project. The current release already represents the largest public domain transcribed voice dataset, with more than 2,400 hours of voice data and 28 languages represented.
(2) In addition to the data collection, Mozilla’s Machine Learning Group has applied sophisticated machine learning techniques and a variety of innovations to build an open-source speech-to-text engine that approaches human accuracy, as well as a text-to-speech engine. Together with the growing Common Voice dataset Mozilla believes this technology can and will enable a wave of innovative products and services, and that it should be available to everyone.
And this is exactly where this new “Mozilla Voice Challenge” fits in: Its objective is to better define the voice technology space by creating a “stack” of open source technologies to support the development of new voice-enabled products.
Stacking the Odds
For the purpose of this competition, we define voice-enabled technologies as technologies that use voice as an interface, allowing people to interact with various connected devices through verbal means — both when speaking and listening.
We envision that some elements of this stack would be the following technologies:
- Speech-to-text (STT)
- Text-to-speech (TSS)
- Natural Language Processing (NLP)
- Voice-signal processing
- Keyword spotting
- Keyword alignment
- Intent parsing
- Language parsing: stemming, entity recognition, dialog management, and summation.
We want to improve this list by adding more relevant technologies and also identify any “gaps” in the stack where quality open source projects are not available (see the Challenge description for more details). We’ll then place the updated list in a public repository for open access — and to achieve this, all proposed technologies in the stack need to be open source licensed.
How to Participate
The competition was posted to the HeroX platform. The competition will run until August 20, 2019 and the submitted proposals will be evaluated by the members of Mozilla’s Voice team. Up to $6,000 in prizes will be awarded to the best proposals.
The challenge is open to everyone (except for Mozilla employees and their families), and we especially encourage members of Mozilla’s Common Voice community to take part in it.
Whether it’s from doing things like burning fossil fuels through driving, cranking up the furnace or grilling a steak, we are all responsible for releasing carbon dioxide into the atmosphere, … Read more
The post Eight ways to reduce your digital carbon footprint appeared first on The Firefox Frontier.
Wireless Routers
In April we announced that the Mozilla IoT team had been working on evolving WebThings Gateway into a full software distribution for consumer wireless routers.
Today, with the 0.9 release, we’re happy to announce the availability of the first experimental builds for our first target router hardware, the Turris Omnia.
Turris Omnia wireless router. Source: turris.cz
These builds are based on the open source OpenWrt operating system. They feature a new first-time setup experience which enables you to configure the gateway as a router and Wi-Fi access point itself, rather than connecting to an existing Wi-Fi network.
Router first time setup
So far, these experimental builds only offer extremely basic router configuration and are not ready to replace your existing wireless router. This is just our first step along the path to creating a full software distribution for wireless routers.
Router network settings
We’re planning to add support for other wireless routers and router developer boards in the near future. We want to ensure that the user community can access a range of affordable developer hardware.
Raspberry Pi 4
As well as these new OpenWrt builds for routers, we will continue to support the existing Raspbian-based builds for the Raspberry Pi. In fact, the 0.9 release is also the first version of WebThings Gateway to support the new Raspberry Pi 4. You can now find a handy download link on the Raspberry Pi website.
Raspberry Pi 4 Model B. Source: raspberrypi.org
Notifier Add-ons
Another feature landing in the 0.9 release is a new type of add-on called notifier add-ons.
Notifier Add-ons
In previous versions of the gateway, the only way you could be notified of events was via browser push notifications. Unfortunately, this is not supported by all browsers, nor is it always the most convenient notification mechanism for users.
A workaround was available by creating add-ons with basic “send notification” actions to implement different types of notifications. However, these required the user to add “things” to their gateway which didn’t represent actual devices and actions had to be hard-coded in the add-on’s configuration.
To remedy this, we have introduced notifier add-ons. Essentially, a notifier creates a set of “outlets”, each of which can be used as an output for a rule. For example, you can now set up a rule to send you an SMS or an email when motion is detected in your home. Notifiers can be configured with a title, a message and a priority level. This allows users to be reached where and how they want, with a message and priority that makes sense to them.
Several planned code deprecations for Firefox 70, currently available on the Nightly pre-release channel, may impact extension and theme developers. Firefox 70 will be released on October 22, 2019.
Aliased theme properties to be removed
In Firefox 65, we started deprecating the aliased theme properties accentcolor, textcolor, and headerURL. These properties will be removed in Firefox 70.
Themes listed on addons.mozilla.org (AMO) will be automatically updated to use supported properties. Most themes were updated back in April, but new themes have been created using the deprecated properties. If your theme is not listed on AMO, or if you are the developer of a dynamic theme, please update your theme’s manifest.json to use the supported properties.
- For
accentcolor, please useframe - For
headerURL, please usetheme_frame - For
textcolor, please usetab_background_text
JavaScript deprecations
In Firefox 70, the non-standard, Firefox-specific Array generic methods introduced with JavaScript 1.6 will be considered deprecated and scheduled for removal in the near future. For more information about which generics will be removed and suggested alternatives, please see the Firefox Site Compatibility blog.
The Site Compatibility working group also intends to remove the non-standard prototype toSource and uneval by the end of 2019.
The post Upcoming deprecations in Firefox 70 appeared first on Mozilla Add-ons Blog.
https://blog.mozilla.org/addons/2019/07/25/upcoming-deprecations-in-firefox-70/
For the last year, Mozilla has been looking for ways to empower voters in light of the shifts in election dynamics caused by the internet and online advertising. This work included our participation in the EU’s Code of Practice on Disinformation to push for change in the industry which led to the launch of the Firefox EU Elections toolkit that provided people information on the voting process, how tracking and opaque online advertising influence their voting behavior and how they can easily protect themselves.
We also had hoped to lend our technical expertise to create an analysis dashboard that would help researchers and journalists monitor the elections. The dashboard would gather data on the political ads running on various platforms and provide a concise “behind the scenes” look at how these ads were shared and targeted.
But to achieve this we needed the platforms to follow through on their own commitment to make the data available through their Ad Archive APIs.
Here’s what happened.
Platforms didn’t supply sufficient data
On March 29, Facebook began releasing its political ad data through a publicly available API. We quickly concluded the API was inadequate.
- Targeting information was not available.
- Bulk data access was not offered.
- Data wasn’t tagged properly.
- Identical searches would produce wildly differing results.
The state of the API made it nearly impossible to extract the data needed to populate the dashboard we were hoping to create to make this information more accessible.
And although Google didn’t provide the targeting criteria advertisers use on the platform, it did provide access to the data in a format that allowed for real research and analysis.
That was not the case for Facebook.
So then what?
It took the entire month of April to figure out ways to work within or rather, around, the API to collect any information about the political ads running on the Facebook platform.
After several weeks, hundreds of hours, and thousands of keystrokes, the Mozilla team created the EU Ad Transparency Reports. The reports contained aggregated statistics on spending and impressions about political ads on Facebook, Instagram, Google, and YouTube.
While this was not the dynamic tool we had envisioned at the beginning of this journey, we hoped it would help.
But despite our best efforts to help Facebook debug their system, the API broke again from May 18 through May 26, making it impossible to use the API and generate any reports in the last days leading up to the elections.
All of this was documented through dozens of bug reports provided to Facebook, identifying ways the API needed to be fixed.
A Roadmap for Facebook
Ultimately our contribution to this effort ended up looking very different than what we had first set out to do. Instead of a tool, we have detailed documentation of every time the API failed and every roadblock encountered and a series of tips and tricks to help others use the API.
This documentation provides Facebook a clear roadmap to make the necessary improvements for a functioning and useful API before the next election takes place. The EU elections have passed, but the need for political messaging transparency has not.
In fact, important elections are expected to take place almost every month until the end of the year and Facebook has recently rolled this tool out globally.
We need Facebook to be better. We need an API that actually helps – not hinders – researchers and journalists uncover who is buying ads, the way these ads are being targeted and to whom they’re being served. It’s this important work that informs the public and policymakers about the nature and consequences of misinformation.
This is too important to get wrong. That is why we plan to continue our work on this matter and continue to work with those pushing to shine
A key theme of the Rust 2019 roadmap is maturity. This covers a variety of topics, but a crucial one is compile times. For example, the roadmap itself has the following as the first main theme for the compiler team.
Improving “core strength” by lowering raw compilation times and also generating better code (which in turn can help with compilation times)
The roadmap explainer post has a “polish” section that has the following as the first example.
Compile times and IDE support
I previously wrote about one period of improvement in Rust compiler speed. How are things going in 2019?
Speed improvements in 2019
The following image shows changes in time taken to compile the standard benchmarks used on the Rust performance tracker. It compares the compiler from 2019-01-01 with the compiler from 2019-07-24 (the most recent data at the time of writing).
These are the wall-time results for 29 benchmarks. There are three different build kinds measured for each one: a debug build, an optimized build, and a check build (which detects errors but doesn’t generate code). For each build kind there is a mix of incremental and non-incremental runs done. The numbers for the individual runs aren’t shown here but you can see them if you view the results directly on the site and click around. The “avg” column shows the average change for those runs. The “min” and “max” columns show the minimum and maximum changes among those same runs.
The table has 261 numbers. The thing to take away is that 258 of them are negative, representing a decrease in compile time. Most of the “avg” values are in the range -20% to -40%. The “min” values (representing the best time reduction for each build kind) range from -12.4% to -51.3%. Even the “max” values (representing the worst time reduction for each build kind) are mostly better than -10%. These are pleasing results.
speed improvements since late 2017
What happens if we look further back? The image below compares the compiler from 2017-11-12 (the earliest date for which I could get data from the site) against the compiler from 2019-07-24, a period of just over 20 months.
These are the wall-time results for only 18 benchmarks, because the benchmark suite was smaller in late 2017. Check builds were also not measured then. You can view the results directly on the site.
My initial thought from looking at the “avg” results was “the compiler is twice as fast” but closer inspection shows that’s not quite true; the average “avg” result is 42%. (I know that averaging averages is statistically dubious, I did it just to get a rough feel.) Overall, the results are significantly better than those for 2019: the “avg” values range from -19.9% to -61.3%, and the “min” values are mostly better than -60%.
(And don’t forget that time reduction percentages can be misleading when they get large. A 50% time reduction means the compiler is twice as fast; a 75% time reduction means the compiler is four times as fast; a 90% time reduction means the compiler is ten times as fast.)
All this is good news. The Rust compiler has long had a reputation for being slow. I still wouldn’t describe it as fast, but it is clearly a lot faster than it used to be. Many thanks to all those who made this happen, and I would be happy to hear from anyone who wants to help continue the
Hello and welcome to another issue of This Week in Rust! Rust is a systems language pursuing the trifecta: safety, concurrency, and speed. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.
This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.
Updates from Rust Community
News & Blog Posts
- U.S. House Committee on Financial Services hearing: Why was the Rust language chosen?
- Microsoft Security Response Center: Why Rust for safe systems programming.
- Compiler speed has improved 30-40% across the board year-to-date, with some projects seeing 45%+ improvements.
- Perils of constructors.
- Notes on a smaller Rust.
- Announcing heim project.
- Gotham — from start to Heroku.
- How to write full stack Rust code.
- HOWTO: Replace mem::uninitialized with mem::MaybeUninit.
- Stream combinators implemented using for await syntax.
- CLion Rust plugin adds a new experimental macro expansion engine.
Crate of the Week
This week's crate is abscissa, a security-oriented Rust application framework. Thanks to Tony Arcieri for the suggestion!
Submit your suggestions and votes for next week!
Call for Participation
Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!
Some of these tasks may also have mentors available, visit the task page for more information.
If you are a Rust project owner and are looking for contributors, please submit tasks here.
Updates from Rust Core
230 pull requests were merged in the last week
- The essence of lexer
- Implement RFC #2203
- Add meta-variable checks in macro definitions
- Stabilize
<*mut _>::castand<*const _>::cast - Specific error for positional args after named args in
format!() - rustc_typeck: Improve diagnostics for
-> _ fn
The first curl release ever saw the light of day on March 20, 1998 and already then, curl could transfer any amount of URLs given on the command line. It would iterate over the entire list and transfer them one by one.
Not even 22 years later, we introduce the ability for the curl command line tool to do parallel transfers! Instead of doing all the provided URLs one by one and only start the next one once the previous has been completed, curl can now be told to do all of them, or at least many of them, at the same time!
This has the potential to drastically decrease the amount of time it takes to complete an operation that involves multiple URLs.
–parallel / -Z
Doing transfers concurrently instead of serially of course changes behavior and thus this is not something that will be done by default. You as the user need to explicitly ask for this to be done, and you do this with the new –parallel option, which also as a short-hand in a single-letter version: -Z (that’s the upper case letter Z).
Limited parallelism
To avoid totally overloading the servers when many URLs are provided or just that curl runs out of sockets it can keep open at the same time, it limits the parallelism. By default curl will only try up to 50 transfers concurrently, so if there are more transfers given to curl those will wait to get started once one of the first transfers are completed. The new –parallel-max command line option can be used to change the concurrency limit.
Progress meter
Is different in this mode. The new progress meter that will show up for parallel transfers is one output for all transfers.
Transfer results
When doing many simultaneous transfers, how do you figure out how they all did individually, like from your script? That’s still to be figured out and implemented.
No same file splitting
This functionality makes curl do URLs in parallel. It will still not download the same URL using multiple parallel transfers the way some other tools do. That might be something to implement and offer in a future fine tuning of this feature.
libcurl already do this fine
This is a new command line feature that uses the fact that libcurl can already do this just fine. Thanks to libcurl being a powerful transfer library that curl uses, enabling this feature was “only” a matter of making sure libcurl was used in a different way than before. This parallel change is entirely in the command line tool code.
Ship
This change has landed in curl’s git repository already (since b8894085000) and is scheduled to ship in curl 7.66.0 on September 11, 2019.
I hope and expect us to keep improving parallel transfers further and we welcome all the help we can get!
Someone forwarded me a MacRumours post that a couple of the (useless) telemetry options in TenFourFox managed to escape my notice and should be disabled. This is true and I'll be flagging them off in FPR16. However, another source of slowdowns popped up recently and while I think it's been pointed out it bears repeating.
On startup, and to a lesser extent when browsing, TenFourFox (and Firefox) enumerates the fonts you have installed on your Power Mac so that sites requesting them can use locally available fonts and not download them unnecessarily. The reason for periodically rechecking is that people can, and do, move fonts around and it would be bad if TenFourFox had stale font information particularly for commonly requested ones. To speed this up, I actually added a TenFourFox-specific font directory cache so that subsequent enumerations are quicker. However, the heuristic for determining when fonts should be rescanned is imperfect and when in doubt I always err towards a fresh scan. That means a certain amount of work is unavoidable under normal circumstances.
Thus, the number of fonts you have currently installed directly affects TenFourFox's performance, and TenFourFox is definitely not the only application that needs to know what fonts are installed. If you have a large (as in several hundred) number of font files and particularly if you are not using an SSD, you should strongly consider thinning them out or using some sort of font management system. Even simply disabling the fonts in Font Book will help, because under the hood this will move the font to a disabled location, and TenFourFox and other applications will then not have to track it further.
How many is too many? On my quad G5, I have about 800 font files on my Samsung SSD. This takes about 3-4 seconds to initially populate the cache and then less than a second on subsequent enumerations. However, on a uniprocessor system and especially on systems without an SSD, I would strongly advise getting that number down below one hundred. Leave the fonts in /System/Library/Fonts alone, but on my vanilla Tiger Sawtooth G4 server, /Library/Fonts has just 87 files. Use Font Book to enable fonts later if you need them for stuff you're working on, or, if you know those fonts aren't ever being used, consider just deleting them entirely.
Due to a work crunch I will not be doing much work on FPR16 until August. However, I will be at the Vintage Computer Festival West again August 3 and 4 at the Computer History Museum in Mountain View. I've met a few readers of this blog in past years, and hopefully getting to play with various PowerPC (non-Power Mac), SPARC and PA-RISC laptops and portable workstations will tempt the rest of you. Come by, say hi, and play around a bit with the other great exhibits that aren't as cool as mine.
http://tenfourfox.blogspot.com/2019/07/clean-out-your-fonts-people.html
The Celery send_task method allows you to invoke a task by name without importing it. [1] There is an undocumented [2] caveat to using send_task: it doesn’t have access to the configuration of the task (from when the task was created using the @task decorator).
Much of this configuration …
https://patrick.cloke.us/posts/2019/07/17/celery-without-a-results-backend/
Last week it seemed that all our limited resource machines were perpetually backlogged. I wrote yesterday to provide insight into what we run and some of our limitations. This post will be discussing the Android phones backlog last week specifically.
The Android phones are hosted at Bitbar and we split them into pools (battery testing, unit testing, perf testing) with perf testing being the majority of the devices.
There were 6 fixes made which resulted in significant wins:
- Recovered offline devices at Bitbar
- Restarting host machines to fix intermittent connection issues at Bitbar
- Update Taskcluster generic-worker startup script to consume superceded jobs
- Rewrite the scheduling script as multi-threaded and utilize bitbar APIs more efficiently
- Turned off duplicate jobs that were on by accident last month
- Removed old taskcluster-worker devices
On top of this there are 3 future wins that could be done to help future proof this:
- upgrade android phones from 8.0 -> 9.0 for more stability
- Enable power testing on generic usb hubs rather than special hubs which require dedicated devices.
- merge all separate pools together to maximize device utilization
With the fixes in place, we are able to keep up with normal load and expect that future spikes in jobs will be shorter lived, instead of lasting an entire week.
Recovered offline devices at Bitbar
Every day a 2-5 devices are offline for some period of time. The Bitbar team finds some on their own and resets the devices, sometimes we notice them and ask for the devices
to be reset. In many cases the devices are hung or have trouble on a reboot (motivation for upgrading to 9.0). I will add to this that the week prior things started getting sideways and it was a holiday week for many, so less people were watching things and more devices ended up in various states.
In total we have 40 pixel2 devices in the perf pool (and 37 Motorola G5 devices as well) and 60 pixel2 devices when including the unittest and battery pools. We found that 19 devices were not accepting jobs and needed attention Monday July 8th. For planning purposes it is assumed that 10% of the devices will be offline, in this case we had 1/3 of our devices offline and we were doing merge day with a lot of big pushes running all the jobs.
Restarting host machines to fix intermittent connection issues at Bitbar
At Bitbar we have a host machine with 4 or more docker containers running and each docker container runs Linux with the Taskcluster generic-worker and the tools to run test jobs. Each docker container is also mapped directly to a phone. The host machines are rarely rebooted and maintained, and we noticed a few instances where the docker containers had trouble connecting to the network. A fix for this was to update the kernel and schedule periodic reboots.
Update Taskcluter generic-worker startup script
When the job is superseded, we would shut down the Taskcluster generic-worker, the docker image, and clean up. Previously it would terminate the job and docker container and then wait for another job to show up (often a 5-20 minute cycle). With the changes made Taskcluster generic-worker will just restart (not docker container) and quickly pick up the next job.
Rewrite the scheduling script as multi-threaded
This was a big area of improvement that was made. As our jobs increased in volume and had a wider range of runtimes, our tool for scheduling was iterating through the queue and devices and calling the APIs at Bitbar to spin up a worker and hand off a task. This is something that takes a few seconds per job or device and with 100 devices it could take 10+ minutes to come around and schedule a new job on a device. With changes made last week ( Bug 1563377 ) we now have jobs starting quickly <10 seconds, which greatly increases our device utilization.
Turn off duplicate opt jobs and only run PGO jobs
In reviewing what was run by default per push and on try, a big oversight was discovered. When we turned PGO on for Android, all the perf jobs were scheduled both for opt and PGO, when they should have been only scheduled for PGO. This was an easy fix and cut a large portion of the load down (