To provide an answer to the question "How many projects use more than one license?"

[fernandocastor]

We need to implement and test the features required to use Groundhog to answer the question in the title of the issue. We then have to use it to actually answer the question.

[rodrigoalvesvieira]

Hmmm,

it brings another vital question to the table: How can we recognize the license(s) of a project? GitHub provides no metadata about it, in fact, GitHub does not even know what License a given project uses.

We have to design some heuristic/technique to grasp this information. The possible approaches that come to my mind are to look for a LICENSE file in the root of the project directory or look for a "License" section in the README.

Then we scrap the text and look for patterns which will allow us to determine which License it is about.

[fernandocastor]

I agree. How hard would it be to implement one such heuristic and how comprehensive would the search be?

[dnr2]

Hi @gustavopinto , how to you plan to extract and discover the license of the project files? Are you planing to use something like FOSSology? I've started studying it to see how we could use their system with groundhog. (maybe you already know that)

[gustavopinto]

I don't know yet how FOSSology works. But, I think that a good start point could be analyzing all text files on the root directory. We then read all of them, and thus perform a string search looking for words like "license", "copy[right|left]" or similar ones. But I still don't know how to discover what is the exact license on the text. :P

[gustavopinto]

Maybe we can catalog all well-known license names, and thus try to find these names on the text files (only in files which exist a 'license-or-similar-word' in the text). We then must have two more license categories: unlincesed (a project which no text file with license-or-similar-word was found) and not-understandable-license (a project which we actually found a text with a license word, but we were not able to discover which license is). What do you say @dnr2?

[fernandocastor]

I think using FOSSology, or at least trying to use it, would be a better idea. Why duplicate the effort that someone else has already spent? Someone has actually used it to answer precisely that question in the recent past: http://www.theregister.co.uk/2013/04/18/github_licensing_study/

[gustavopinto]

Yes, you are right. Let's give FOSSology a chance.

[gustavopinto]

Here I'm again. I think it will be very hard to integrate FOSSology into groundhog, mainly because FOSSology is written in php 👎

Do any of you know other alternatives?

[fernandocastor]

But is it a library or a Web page/service?

[dnr2]

Both!

I've tested their demo system on [1] but it seems to be only a web based service (I couldn't find any API). So I believe that if we want to use FOSSology we only have two options: either we run our own FOSSology server or we could rely on their web based demo system.

IMO we should follow @gustavopinto 's first idea and develop some kind of algorithm to discover the project's licence, unless we find another FOSSology server.

[1] - https://fossology.ist.unomaha.edu/
[2] - http://www.fossology.org/projects/fossology/wiki/Live_Demo

[dnr2]

You can test their system in: https://fossology.ist.unomaha.edu/?mod=agent_nomos_once

Just create a license text file and upload, it really works!

[gustavopinto]

yeah, @dnr2 is right.

[rodrigoalvesvieira]

We are then set to discuss how we can develop this algorithm for identifying licenses. To begin the discussion, IMO there are tree places in a GitHub project to search for the license text:

1. LICENSE file. Many projects have a LICENSE txt or markdown file in the source code root where the license text is present. I think this is the optimal situation and - although not so mainstream - it's quite popular and a lot of projects that we'll search will have this characteristic.
2. License text/declaration in the README file. README files are widely adopted by projects on GitHub (as in elsewhere). I have observed that in many projects, the LICENSE file is missing but the README contains the project's legal information.
3. License text within the source code. Now this is a fairly adopted practice (specially among popular projects) of having the license written down in some (or many) files in the source code. This is certainly the worst case because we could end up having to look up too many files. Fortunately, this is to be done locally so in no way it could drain more of our API request rates.

BTW, this is another limitation of GitHub. The service does not formally collect licensing information for its hosted projects.

This is based in my experience using GitHub, which is of course subjected to my personal biases. But I think it's a good starting point anyway. Before figuring out how we will identify/classify the licensing data, we must know how we will obtain it.

Bring your ideas onto the table!

-- rodrigo

On Jul 12, 2013, at 7:41 PM, Gustavo notifications@github.com wrote:

yeah, @dnr2 is right.

—
Reply to this email directly or view it on GitHub.