The Apache Software Foundation
The Apache Software Foundation Incubator

Spark Incubation Status

This page tracks the project status, incubator-wise. For more general project status, look on the project website.

The Spark project graduated on 2014-02-19


The Apache Spark is an open source system for fast and flexible large-scale data analysis. Spark provides a general purpose runtime that supports low-latency execution in several forms. These include interactive exploration of very large datasets, near real-time stream processing, and ad-hoc SQL analytics (through higher layer extensions).


Project info

item type reference
Website www
. wiki .
Mailing list dev
. commits
Bug tracking .
Source code Git
Mentors mattmann Chris Mattmann
. pramirez Paul Ramirez
. ahart Andrew Hart
. tomdz Thomas Dudziak
. smarru Suresh Marru
. hsaputra Henry Saputra
. rvs Roman Shaposhnik
Committers matei Matei Zaharia
. ankurdave Ankur Dave
. tdas Tathagata Das
. haoyuan Haoyuan Li
. joshrosen Josh Rosen
. rxin Reynold Xin
. shivaram Shivaram Venkataraman
. mosharaf Mosharaf Chowdhury
. woggle Charles Reiss
. andrew Andy Konwinski
. pwendell Patrick Wendell
. irashid Imran Rashid
. ryanlecompte Ryan LeCompte
. harsha Ram Sriharsha
. bobby Robert Evans
. mridulm80 Mridul Muralidharan
. tomdz Thomas Dudziak
. markhamstra Mark Hamstra
. stephenh Stephen Haberman
. jasondai Jason Dai
. shane_huang Shane Huang
. xiajunluan Andrew Xia
. mlnick Nick Pentreath
. mcnamara Sean McNamara
. prashant Prashant Sharma
. tgraves Thomas Graves
. kayousterhout Kay Ousterhout
. adav Aaron Davidson
Extra . .

Incubation status reports

Incubation work items

Project Setup

This is the first phase on incubation, needed to start the project at Apache.

Item assignment is shown by the Apache id. Completed tasks are shown by the completion date (YYYY-MM-dd).

Identify the project to be incubated

date item
2013-06-19 Make sure that the requested project name does not already exist. Please follow the guide to ensure a suitable project/product name.
N/A If request from an existing Apache project to adopt an external package, then ask the Apache project for the SVN module and mail address names.
2013-06-01 If request from outside Apache to enter an existing Apache project, then post a message to that project for them to decide on acceptance.
2013-06-19 If request from anywhere to become a stand-alone PMC, then assess the fit with the ASF, and create the lists and modules under the incubator address/module names if accepted.


date item
2013-06-19 Ask infrastructure to create source repository modules and grant the committers karma.
2013-06-19 Ask infrastructure to set up and archive mailing lists.
2013-06-19 Ask infrastructure to set up issue tracker (JIRA, Bugzilla).
2013-06-29. Ask infrastructure to set up wiki (Confluence, Moin).
2013-06-19 Migrate the project to our infrastructure.

Mentor-related responsibility/oversight

date item
2013-06-19 Identify all the Mentors for the incubation, by asking all that can be Mentors.
2014-01-30 Subscribe all Mentors on the pmc and general lists.
2014-01-30 Give all Mentors access to the incubator SVN repository. (to be done by the Incubator PMC chair or an Incubator PMC Member wih karma for the authorizations file)
2013-06-19 Tell Mentors to track progress in the file 'incubator/projects/{}.html'
date item
2013-06-20 Check and make sure that the papers that transfer rights to the ASF been received. It is only necessary to transfer rights for the package, the core code, and any new code produced by the project.
2014-01-30 Check and make sure that the files that have been donated have been updated to reflect the new ASF copyright.

Verify distribution rights

date item
2014-01-30 Check and make sure that for all code included with the distribution that is not under the Apache license, we have the right to combine with Apache-licensed code and redistribute.
2014-01-30 Check and make sure that all source code distributed by the project is covered by one or more of the following approved licenses: Apache, BSD, Artistic, MIT/X, MIT/W3C, MPL 1.1, or something with essentially the same terms.

Establish a list of active committers

date item
2014-01-30 Check that all active committers have submitted a contributors agreement.
2014-01-30 Add all active committers in the STATUS file.
2014-01-30 Ask root for the creation of committers' accounts on

Project specific

Add project specific tasks here.


These action items have to be checked for during the whole incubation process.

These items are not to be signed as done during incubation, as they may change during incubation. They are to be looked into and described in the status reports and completed in the request for incubation signoff.

Collaborative Development

  • Have all of the active long-term volunteers been identified and acknowledged as committers on the project?
  • Are there three or more independent committers? (The legal definition of independent is long and boring, but basically it means that there is no binding relationship between the individuals, such as a shared employer, that is capable of overriding their free will as individuals, directly or indirectly.)
  • Are project decisions being made in public by the committers?
  • Are the decision-making guidelines published and agreed to by all of the committers?

Licensing awareness

  • Are all licensing, trademark, credit issues being taken care of and acknowleged by all committers?

Project Specific

Add project specific tasks here.


Things to check for before voting the project out.

Organizational acceptance of responsibility for the project

date item
2014-02-15 If graduating to an existing PMC, has the PMC voted to accept it? see here
2014-02-19 If graduating to a new PMC, has the board voted to accept it? see here

Incubator sign-off

  • Has the Incubator decided that the project has accomplished all of the above tasks?