
Last week, the team was in San Francisco for an all-Mozilla company meeting.
This week the team is focusing on adding new features, making improvements and fixing bugs.
We are all hands on deck building more components and adding new UI across Firefox Reality:
Here is a preview that we showed off of the support for skybox and some of the new UX/UI:
We are continuing to provide a better experience across Hubs by Mozilla:
Join our public WebVR Slack #social channel to participate in on the discussion!
Found a critical bug? File it in our public GitHub repo or let us know on the public WebVR Slack #unity channel and as always, join us in our discussion!
Stay tuned for new features and improvements across our three areas!
On 20 June the European Parliament’s legal affairs committee (JURI) approved its report on the copyright directive, sending the controversial and dangerous copyright reform into its final stages of lawmaking.
Here is a statement from Raegan MacDonald, Mozilla’s Head of EU Public Policy:
“This is a sad day for the Internet in Europe. Lawmakers in the European Parliament have just voted for a new law that would effectively impose a universal monitoring obligation on users posting content online. As bad as that is, the Parliament’s vote would also introduce a ‘link tax’ that will undermine access to knowledge and the sharing of information in Europe.
It is especially disappointing that just a few weeks after the entry into force of the GDPR – a law that made Europe a global regulatory standard bearer – Parliamentarians have approved a law that will fundamentally damage the Internet in Europe, with global ramifications. But it’s not over yet – the final text still needs to be signed off by the Parliament plenary on 4 July. We call on Parliamentarians, and all those who care for an internet that fosters creativity and competition in Europe, to overturn these regressive provisions in July.”
Article 11 – where press publishers can demand a license fee for snippets of text online – passed by a slim majority of 13 to 12. The provision mandating upload filters for copyright content, Article 13, was adopted 15 to 10.
Mozilla will continue to fight for copyright that suits the 21st century and fosters creativity and competition online. We encourage anyone who shares these concerns to reach out to members of the European Parliament – you can call them directly via changecopyright.org, or tweet and email them at saveyourinternet.eu.
The post Parliament adopts dangerous copyright proposal – but the battle continues appeared first on Open Policy & Advocacy.
Please join us in congratulating Prathamesh Chavan, our Rep of the Month for May 2018!
Prathamesh is from Pune, India and works as a Technical Support Engineer at Red Hat. From his very early days in the Mozilla community, Prathamesh used his excellect people skills to spread the community to different colleges and to evangelise many of the upcoming projects, products and Mozilla initiatives. Prathamesh is also a very resourceful person. Due to this, he did a great job at organizing some great events at Pune and creare many new Mozilla Clubs across the city there.
As a Mozilla Reps Council member, Prathamesh has done some great work and has shown great leadership skills. He is always proactive in sharing important updates with the bigger community as well as raising his hand at every new initiative.
Thanks Prathamesh, keep rocking the Open Web!
Please congratulate him by heading over to the Discourse topic.
https://blog.mozilla.org/mozillareps/2018/06/22/rep-of-the-month-may-2018/
Firefox is one of the world’s most successful open source software projects. This means we make the code that runs Firefox available for anyone to modify and use so long … Read more
The post Open source isn’t just for software: Opensourcery recipe appeared first on The Firefox Frontier.
https://blog.mozilla.org/firefox/open-source-isnt-just-for-software-opensourcery-recipe/
async with aiohttp.ClientSession() as session:
async with async_timeout.timeout(seconds_for_timeout):
async with session.put(
"http://gateway.local/things/{}/properties/color".format(thing_id),
headers={
'Accept': 'application/json',
'Authorization': 'Bearer {}'.format(things_gateway_auth_key),
'Content-Type': 'application/json'
},
data='{{"color": "{}"}}'.format(a_color)
) as response:
return await response.text()
Last week, more than 1,200 Mozillians from around the globe converged on San Francisco, California, for Mozilla’s biannual All Hands meeting to celebrate recent successes, learn more about products from around the company, and collaborate on projects currently in flight.
For the add-ons team, this meant discussing tooling improvements for extension developers, reviewing upcoming changes to addons.mozilla.org (AMO), sharing what’s in store for the WebExtensions API, and checking in on initiatives that help users discover extensions. Here are some highlights:
During a recent survey, participating extension developers noted two stand-out tools for development: web-ext, a command line tool that can run, lint, package, and sign an extension; and about:debugging, a page where developers can temporarily install their extensions for manual testing. There are improvements coming to both of these tools in the coming months.
In the immediate future, we want to add a feature to web-ext that would let developers submit their extensions to AMO. Our ability to add this feature is currently blocked by how AMO handles extension metadata. Once that issue is resolved, you can expect to see web-ext support a submit command. We also discussed implementing a create command that would generate a standard extension template for developers to start from.
Developers can currently test their extensions manually by installing them through about:debugging. Unfortunately, these installations do not persist once the browser is closed or restarted. Making these installations persistent is on our radar, and now that we are back from the All Hands, we will be looking at developing a plan and finding resources for implementation.
During the next three months, the AMO engineering team will prioritize work around improving user rating and review flows, improving the code review tools for add-on reviewers, and converting dictionaries to WebExtensions.
Engineers will also tackle a project to ensure that users who download Firefox because they want to install a particular extension or theme from AMO are able to successfully complete the installation process. Currently, users who download Firefox from a listing on AMO are not returned to AMO when they start Firefox for the first time, making it hard for them to finish installing the extension they want. By closing this loop, we expect to see an increase in extension and/or theme installations.
Several new and enhanced APIs have landed in Firefox since January, and more are on their way. In the next six months, we anticipate landing WebExtensions APIs for clipboard support, bookmarks and session management (including bookmark tags and further expansions of the theming API).
Additionally, we’ll be working towards supporting visual overlays (like notification bars, floating panels, popups, and toolbars) by the end of the year.
This year, we are focusing on helping Firefox users find and discover great extensions quickly. We have made a few bets on how we can better meet user needs by recommending specific add-ons. In San Francisco, we checked in on the status of projects currently underway:
In May, we started testing recommendations on listing pages for extensions commonly co-installed by other users.
Results so far have shown that people are discovering and installing more relevant extensions from these recommendations than the control group, who only sees generally popular extensions. We will continue to make refinements and fully graduate it into AMO in the second half of the year.
(For our privacy-minded friends: you can
This is a weekly call with some of the Reps to discuss all matters about/affecting Reps and invite Reps to share their work with everyone.
mconley livehacks on real Firefox bugs while thinking aloud.
| Project | What’s in it? | Status |
| C++17 | See list | Published! |
| C++20 | See below | On track |
| Library Fundamentals TS v2 | source code information capture and various utilities | Published! Parts of it merged into C++17 |
| Concepts TS | Constrained templates | Merged into C++20 with some modifications |
| Parallelism TS v2 | Task blocks, library vector types and algorithms, and more | Approved for publication! |
| Transactional Memory TS | Transaction support | Published! Not headed towards C++20 |
| Concurrency TS v1 | future.then(), latches and barriers, atomic smart pointers |
Published! Parts of it merged into C++20, more on the way |
| Executors | Abstraction for where/how code runs in a concurrent context | Final design being hashed out. Ship vehicle not decided yet. |
| Concurrency TS v2 | See below | Under development. Depends on Executors. |
| Networking TS | Sockets library based on Boost.ASIO | Published! |
| Ranges TS | Range-based algorithms and views | Published! Headed towards C++20 |
| Coroutines TS | Resumable functions, based on Microsoft’s await design |
Published! C++20 merge uncertain |
| Modules v1 | A component system to supersede the textual header file inclusion model | Published as a TS |
| Modules v2 | Improvements to Modules v1, including a better transition path | Under active development |
| Numerics TS | Various numerical facilities | Under active development |
| Graphics TS | 2D drawing API | No consensus to move forward |
| Reflection TS | Static code reflection mechanisms | Send out for PDTS ballot |
| Contracts | Preconditions, postconditions, and assertions | Merged into C++20 |
A few links in this blog post may not resolve until the committee’s post-meeting mailing is published (expected within a few days of June 25, 2018). If you encounter such a link, please check back in a few days.
A couple of weeks ago I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in Rapperswil, Switzerland. This was the second committee meeting in 2018; you can find my reports on preceding meetings here (March 2018, Jacksonville) and here (November 2017, Albuquerque), and earlier ones linked from those. These reports, particularly the Jacksonville one, provide useful context for this post.
At this meeting, the committee was focused full-steam on C++20, including advancing several significant features — such as Ranges, Modules, Coroutines, and Executors — for possible inclusion in C++20, with a secondary focus on in-flight Technical Specifications such as the Parallelism TS v2, and the Reflection TS.
C++20 continues to be under active development. A number of new changes have been voted into its Working Draft at this meeting, which I list here. For a list of changes voted in at previous meetings, see my
As of Firefox 60, the build environment for official Firefox Linux builds switched from CentOS to Debian.
As part of the transition, we overhauled how the build environment for Firefox is constructed. We now populate the environment from deterministic package snapshots and are much more stringent about dependencies and operations being deterministic and reproducible. The end result is that the build environment for Firefox is deterministic enough to enable Firefox itself to be built deterministically.
Changing the underlying operating system environment used for builds was a risky change. Differences in the resulting build could result in new bugs or some users not being able to run the official builds. We figured a good way to mitigate that risk was to make the old and new builds as bit-identical as possible. After all, if the environments produce the same bits, then nothing has effectively changed and there should be no new risk for end-users.
Employing the diffoscope tool, we identified areas where Firefox builds weren't deterministic in the same environment and where there was variance across build environments. We iterated on differences and changed systems so variance would no longer occur. By the end of the process, we had bit-identical Firefox builds across environments.
So, as of Firefox 60, Firefox builds on Linux are deterministic in our official build environment!
That being said, the builds we ship to users are using PGO. And an end-to-end build involving PGO is intrinsically not deterministic because it relies on timing data that varies from one run to the next. And we don't yet have continuous automated end-to-end testing that determinism holds. But the underlying infrastructure to support deterministic and reproducible Firefox builds is there and is not going away. I think that's a milestone worth celebrating.
This milestone required the effort of many people, often working indirectly toward it. Debian's reproducible builds effort gave us an operating system that provided deterministic and reproducible guarantees. Switching Firefox CI to Taskcluster enabled us to switch to Debian relatively easily. Many were involved with non-determinism fixes in Firefox over the years. But Mike Hommey drove the transition of the build environment to Debian and he deserves recognition for his individual contribution. Thanks to all these efforts - and especially Mike Hommey's - we can now say Firefox builds deterministically!
The fx-reproducible-build bug tracks ongoing efforts to further improve the reproducibility story of Firefox. (~300 bugs in its dependency tree have already been resolved!)
http://gregoryszorc.com/blog/2018/06/20/deterministic-firefox-builds
In part 2 of this two-part video, managers can use this Playbook to assess their employees' performance and make recommendations about bonus and merit.
https://air.mozilla.org/guidance-for-h1-merit-and-bonus-award-cycle/
In part 1 of this two-part video, managers can use this Playbook to help assess their employees' performance and make bonus and merit recommendations.
https://air.mozilla.org/manager-playbook-performance-assessment/
Mozilla is rolling out Phabricator as part of our tooling. However, at the time of writing I was unable to find a straight forward setup to get the Phabricator tooling playing nice on Windows with MozillaBuild.
Right now there are a couple of separate threads around how to interact with Phabricator on Windows:
However, I have stuff waiting for me on Phabricator that I'd like to interact with now, so let's get a work around in place! I started with the Arcanist windows steps, but have adapted them to a MozillaBuild specific environment.
Arcanist requires PHP. Grab a build from here. The docs for Arcanist indicate the type of build doesn't really matter, but I opted for a thread safe one because that seems like a nice thing to have.
I installed PHP outside of my MozillaBuild directory, but you can put it anywhere. For the sake of example, my install is in C:\Patches\Php\php-7.2.6-Win32-VC15-x64.
We need to enable the curl extension: in the PHP install dir copy php.ini-development to php.ini and uncomment (by removing the ;) the extension=curl line.
Finally, enable PHP to find its extension by uncommenting the extension_dir = "ext" line. The Arcanist instructions suggest setting a fully qualified path, but I found a relative path worked fine.
Create somewhere to store Arcanist and libphutil. Note, these need to be located in the same directory for arc to work.
$ mkdir somewhere/
$ cd somewhere/
somewhere/ $ git clone https://github.com/phacility/libphutil.git
somewhere/ $ git clone https://github.com/phacility/arcanist.git
For me this is C:\Patches\phacility\.
Since I want arc to be usable in MozillaBuild until this work around is no longer required, we're going to modify start up settings. We can do this by changing ...mozilla-build/msys/etc/profile.d and adding to the PATH already being constructed. In my case I've added that paths mentioned earlier, but with MSYS syntax: /c/Patches/Php/php-7.2.6-Win32-VC15-x64:/c/Patches/phacility/arcanist/bin:.
Now arc should run inside newly started MozillaBuild shells.
We still need credentials in order to use arc with mozilla-central. For this to work we need a Phabricator account, see here for that. After that's done, in order to get your credentials run arc install-certificate, navigate to the page as instructed and copy your API key back to the command line.
There was an issue with the evolve Mercurial extension the would cuase Unknown Mercurial log field 'instability'!. This should now be fixed in Arcanist. See this bug for more info.
Finally, I had some issues with arc diff based on my Mercurial config. Updating my extensions and running a ./mach bootsrap seemed to help.
Hopefully everything is ready to go at this point. I found working through the Mozilla docs for how to use arc after setup helpful. If have any comments, please let me know either via email or on IRC.
https://www.brycevandyk.com/setting-up-arcanist-for-mozilla-development-on-windows/

In the last few months, Mozilla has invested in collaboration with other open source project leaders and academics who care about improving diversity & inclusion in Open Source through the CHAOSS D&I working group. Contributors so far include:
Alexander Serebrenik (Eindhoven University of Technology) , Akshita Gupta (Outreachy), Amy Marrich (OpenStack), Anita Sarma (Oregon State University), Bhagashree Uday (Fedora), Daniel Izquierdo (Bitergia), Emma Irwin (Mozilla), Georg Link (University of Nebraska at Omaha), Gina Helfrich (NumFOCUS), Nicole Huesman (Intel) and Sean Goggins ((University of Missouri).
Our goals are to first establish a set of peer-validated goal-metrics, for understanding diversity & inclusion in FOSS ; Second, to identify technology, and research methodologies for understanding the success of our interventions in ways that keep ethics of privacy, and consent at center. And finally, that we document this work in ways that communities can reproduce the report for themselves.
For Mozilla this follows the recommendations coming out of our D&I research to create Metrics that Matter, and to work across Open Source with others projects trying to solve the same problems. I am very excited to share our first draft of goal-metrics for your feedback.

Please note that we know these are incomplete, we know there are likely existing resources that can improve, or even disprove some of these — and that is the point of this blog post! Please review and provide feedback either — via a Github issue, pull request, or by reaching out to someone in the working group, or by joining our working group call (next one July 20th, 9am PST) — which you can find the

Have you ever wanted to make a small web app to share your favorite places with your friends? For example your favorite photographs attached to a hike, or just a view of your favorite peak, or your favorite places downtown, or a suggested itinerary for friends visiting?
Right now it is difficult to easily incorporate third party map data into your own projects. Creating 3d games or VR experiences with real world maps requires access to proprietary software or closed data ecosystems. To do it from scratch requires pulling data from multiple sources, such as image servers, and elevation servers. It also requires substantial math expertise. As well, often you may want to stylize the rendering to suit your own specific use cases. You may have a tron like video game aesthetic for your project and yet the building geometry you're forced to work with doesn't allow you to change colors. While there are many map providers, such as Apple, Google Maps and suchlike, and there are many viewers - most of these tools are specialized around showing high fidelity maps that are as true to reality as possible. What's missing is a middle ground - where you can take map data and easily put it in your own projects - creating your own mash ups.
We see A-Terrain as a starting point or demo for how the web can be different. With this component you can build whatever 3D experience you want and use real world data.
We’ve arranged for Cesium ion (see http://cesium.com) to make the data set available for free for people to try out. Currently the dataset includes world elevation, satellite images and 3d buildings for San Francisco.
For example here is a stylized view of San Francisco as viewed from ground level on the Embarcadero:

You can try this example yourself in your browser here (use AWSD or arrow keys to move around):
https://anselm.github.io/aterrain/examples/helloworld/tile.html .
This component can also be used as a quick and dirty globe renderer (although if you're really interested in that specific use case then Cesium itself may be more suitable):

I have added some rudimentary navigation controls using hash arguments on the URL. For example here is a piece of Mt Whitney:
https://anselm.github.io/aterrain/examples/place/index.html#lat=36.57850&lon=-118.29226&elev=1000

The real strength of a tool like this is composability — to be able to mix different components together. For example here is A-Terrain and Mozilla Hubs being used for a collaborative hiking trip planning scenario to the Grand Canyon:

Here is the URL for the above. This will take you to a random room ID - share that room ID with your friends to join the same room:
As another example of lightweight composability I place a tiny duck on the earths surface above Oregon. This is just a few lines of scripting:

This example can be visited here:
https://anselm.github.io/aterrain/examples/helloworld/duck.html
To accomplish all this we leverage A-Frame — a browser based framework that lets users build 3d environments easily. The A-Frame philosophy is to take complicated behaviors and wrap them up html tags. If you can write ordinary HTML you can build 3d environments.
A-Frame is part of a Mozilla initiative to foster the open web —to raise the bar on what people can create on the web. Using A-Frame anybody can make 3d, virtual or augmented reality experiences on the web. These experiences can be shared instantly with anybody else in the world — running in the browser, on mobile phones, tablets and high end head mounted
So Mozilla is going all-in on Phabricator and Differential as a code review tool. I have mixed feelings on this, not least because it’s support for patch series is more manual than I’d like. But since this is the choice Mozilla has made I might as well start to get used to it. One of the first things you see when you log into Phabricator is a default view full of information.
It’s a little overwhelming for my tastes. The Recent Activity section in particular is more than I need, it seems to list anything anyone has done with Phabricator recently. Sorry Ted, but I don’t care about that review comment you posted. Likewise the Active Reviews section seems very full when it is barely listing any reviews.
But here’s the good news. Phabricator lets you create your own dashboards to use as your default view. It’s a bit tricky to figure out so here is a quick crash course.
Click on Dashboards on the left menu. Click on Create Dashboard in the top right, make your choices then hit Continue. I recommend starting with an empty Dashboard so you can just add what you want to it. Everything on the next screen can be modified later but you probably want to make your dashboard only visible to you. Once created click “Install Dashboard” at the top right and it will be added to the menu on the left and be the default screen when you load Phabricator.
Now you have to add searches to your dashboard. Go to Differential’s advanced search. Fill out the form to search for what you want. A quick example. Set “Reviewers” to “Current Viewer”, “Statuses” to “Needs Review”, then click Search. You should see any revisions waiting on you to review them. Tinker with the search settings and search all you like. Once you’re happy click “Use Results” and “Add to Dashboard”. Give your search a name and select your dashboard. Now your dashboard will display your search whenever loaded. Add as many searches as you like!
Here is my very simple dashboard that lists anything I have to review, revisions I am currently working on and an archive of closed work:
Like it? I made it public and you can see it and install it to use yourself if you like!
https://www.oxymoronical.com/blog/2018/06/Taming-Phabricator
This week I was doing an experiment to see if I could automate a build step in a project I'm working on, which requires binary resources to be included in a web app.
I'm building a custom Linux kernel and bundling it with a root filesystem in order to embed it in the browser. To do this, I'm using a dockerized Buildroot build environment (I'll write about the details of this in a follow-up post). On my various computers, this takes anywhere from 15-25 minutes. Since my buildroot/kernel configs won't change very often, I wondered if I could move this to Travis and automate it away from our workflow?
Travis has no problem using docker, and as long as you can fit your build into the alloted 50 minute build timeout window, it should work. Let's do this!
In the simplest case, doing a build like this would be as simple as:
sudo: required
services:
- docker
...
before_script:
- docker build -t buildroot .
- docker run --rm -v $PWD/build:/build buildroot
...
deploy:
# Deploy built binaries in /build along with other assets
This happily builds my docker buildroot image, and then starts the build within the container, logging everything as it goes. But once the log gets to 10,000 lines in length, Travis won't produce more output. You can still download the Raw Log as a file, so I wait a bit and then periodically download a snapshot of the log in order to check on the build's progress.
At a certain point the build is terminated: once the log file grows to 4M, Travis assumes that all the size is noise, for example, a command running in an infinite loop, and terminates the build with an error.
It's clear that I need to reduce the output of my build. This time I redirect build output to a log file, and then tell Travis to dump the tail-end of the log file in the case of a failed build. The after_failre and after_success build stage hooks are perfect for this.:
before_script:
- docker build -t buildroot . > build.log 2>&1
- docker run --rm -v $PWD/build:/build buildroot >> build.log 2>&1
after_failure:
# dump the last 2000 lines of our build, and hope the error is in that!
- tail --lines=2000 build.log
after_success:
# Log that the build worked, because we all need some good news
- echo "Buildroot build succeeded, binary in ./build"
I'm pretty proud of this until it fails after 10 minutes of building with an error about Travis assuming the lack of log messages (which are all going to my build.log file) means my build has stalled and should be terminated. Turns out you must produce console output every 10 minutes to keep Travis builds alive.
Not only is this a common problem, Travis has a built-in solution in the form of travis_wait. Essentially, you can prefix your build command with travis_wait and it will tolerate there being no output for 20 minutes. Need more than 20, you can optionally pass it the number of minutes to wait before timing out. Let's try 30 minutes:
before_script:
- docker build -t buildroot . > build.log 2>&1
- travis_wait 30 docker run --rm -v $PWD/build:/build buildroot >> build.log 2>&1
This builds perfectly...for 10 minutes. Then it dies with a timeout due to there being no console output. Some more research reveals that travis_wait doesn't play nicely with processes that fork or exec.
Lots of people suggest variations on the same theme: run a command that spins and periodically prints something to stdout, and have it fork your build process:
before_script:
- docker build -t buildroot . > build.log 2>&1
- while sleep 5m; do echo "=====[ $SECONDS seconds, buildroot still building... ]====="; done &
- time docker run --rm -v $PWD/build:/build buildroot >> build.log 2>&1
# Killing background sleep loop
- kill %1
Here we log something at 5 minute intervals, while the build progresses in the background. When it's done,
Newsletter number twenty is here, delayed again by a combination of days off and the bi-annual Mozilla AllHands which took place last week in San Francisco.
A big highlight in the WebRender side is the work on porting all primitives to the brush system approaching completion. Individually, porting each primitive doesn’t sound like much but with all of the pieces coming together:
This translates into noticeable performance imporvements on a lot of very complex pages. The most outstanding remaining performance issues are now caused by the CPU fallback which we are working on moving off of the critical path, so things are looking very promiscing especially with the mountain of other performance improvements we are currently holding off on to concentrate on correctness.
Speaking of fixing correctness issues, as usual we can see from the lists below that there is also a lot of progress in this area.
At last week’s Mozilla All Hands meeting in San Francisco we had an Oxidation meeting about the use of Rust in Firefox. It was low-key, being mostly about status and progress. The notes are here for those who are interested.
https://blog.mozilla.org/nnethercote/2018/06/19/san-francisco-oxidation-meeting-notes/