Re: Security issues in Emacs packages

[Tim Cross <theophilusx@gmail.com>]

Jean Louis <bugs@gnu.support> writes:

> * Tim Cross <theophilusx@gmail.com> [2020-11-25 10:01]:
>>
>> Jean Louis <bugs@gnu.support> writes:
>>
>> > * Tim Cross <theophilusx@gmail.com> [2020-11-24 23:40]:
>> >> If people are really concerned about security, they should look first at
>> >> their use of repositories like MELPA. There is no formal review or
>> >> analysis of packages in these repositories, yet people will happily
>> >> select some package and install it.
>> >
>> > Interesting that you are one who mentions that. There are just few
>> > people ever mentioned it.
>> >
>> > I am still in process of the review of MELPA packages and its
>> > system. There are many security issues.
>> >
>> > Package signing is one example. It does not offer much of security
>> > when packages are signed automatically, but it raises level of
>> > security.
>> >
>> > MELPA packages and archive-contents are not PGP signed, while GNU ELPA
>> > packages are signed.
>> >
>>
>> IMO signing of packages is irrelevant when there is no formal review
>> process or even any formal process to verify the credentials of
>> signatures. In fact, just adding signing would likely be
>> coutner-productive as it would give the impression of some sort of
>> security where there is none.
>
> When user receives a signed package from GNU ELPA, that means it comes
> from GNU ELPA. If signed package is then distributed by mirrors or
> other websites users who enable package signature verifications will
> know that package still comes from GNU ELPA, and not from Chinese
> distributor. If the package is tampered the signature verification
> will not allow installation of the package.
>

I think you missed my point. There is no benefit in MELPA adopting
signed packages because there is no formal code review and no vetting of
the individuals who submit the code. If you have no controls in place
over the contents of what is being signed, the value of the signatures
as a security measure is drastically reduced. Yes, the valid signature
may provide some assurances as to where the package originated, but that
means little if the contents could be anything.

The situation with ELPA is a little better because those who maintain
the code are required to assign legal copyright to GNU. However, I'm not
sure how much checking is done to verify the information in those
assignments. As far as I know, there is no formal code review. A number
of the Emacs developers do perform some informal review, but as we all
know from the issues with openssl, informal reviews provide little
assurance. This is not a criticism of GNU or emacs developers. The
amount of resources necessary to perform formal review is much larger
than the available resources. On the whole, the emacs user community
appears to be happy with the current situation. If they were not, it
would be on the community to step up and do something about it.

As it stands, the signing of ELPA packages only provides assurance that
they are packages assembled by GNU. These signatures do not provide any
real assurance regarding the content of the packages other than they are
GPL's and do not recommend or encourage the use of non-free software.


> That it does add to security shows the fact that GNU/Linux
> distributions do sign packages. There is difference if the package
> comes from trusted source or not trusted source.
>

The question is what level of trust should you assume. With ELPA, all
you can really trust is that the package has a GPL license and does not
recommend/require the use of non-free software. There is little trust
that the package does not do something malicious or includes code which
may compromise the user's security. to provide that level of assurance,
you would need formal code reviews, which is not feasible given
available resources. I think it is important users are aware of this
limitation. Furthermore, I ask the question "Does having signed packages
imply a level of expected assurance which is higher than it really is?"
In other words, do users expect that a package is completely safe just
because it was downloaded from an official GNU ELPA repository?

>> Basically, anyone can upload anything to MELPA.
>
> Maintainers verifies the package initially for certain conventions,
> after that, how I have understood it, packages are automatically
> pulled from Git. The more authors give packages to MELPA the more
> insecurity probability is raised.
>
> GNU ELPA how I understand it, I may be wrong, works like this:
>
> 1) packages are uploaded to GNU ELPA
> 2) then automatically signed by GNU ELPA PGP keys
> 3) offered for distribution
>
Last time I looked, ELPA also supported 'external' packages where the
data is retrieved from an external git repository. I think org is one
such package.

> The point number (1) is human, not automated. Author decides when is
> the package ripe for distribution and what is "release".
>
> Git repository is never release and not meant to be "release". Git is
> for collaborative development and users are made blind that it is some
> kind of release while it is not. One shall always assume that Git
> repository contains development versions not ready for public.
>

Why? This is not normal. Git repositories contain all versions, both
production and development. What is production and what is development
is managed through branches and tags. Anyone who wants can clone the
ELPA git repository.

> MELPA pulls those packages, correct me if I am wrong, automatically
> from Git repositories without regard if the package is actually
> release. That does not align or respect the established Emacs
> conventions how packages should be released, if they are multi files
> they should be in .tar file otherwise .el and there are version
> numbers that MELPA fiddles with and makes possible conflicts and
> introduces confusions.

this is wrong. In melpa you specify either a commit (SHA) or a branch or
both. The repository owner has control over this. MELPA doesn't just
pull data from the repository because there has bene an update. You can
configure things so that whenever data is committed to a release branch,
it is pulled, but this is under the control of the repository owner. It
isn't that different to ELPA where the maintainer will either push new
data to the ELPA repository (or ask someone with write permission to
pull it from their repository).

>
> In GNU ELPA authors decide when package is release ready and when
> it should be released.
>
> In MELPA authors only apply for their packages to be pulled out
> automatically and offered for distribution.
>

You imply authors do not have control over when new releases are made.
This is not the case. They have full control.

> Both repositories could be compromised but probability is becoming
> larger and larger that by automatic pulling of packages something
> worse happens.

The risks between the two systems are not significantly different
because neither system performs any formal review of the code. In some
respects, this is more of an issue for ELPA because there is likely a
higher expectation that the code from ELPA can be trusted more than the
code from MELPA. ELPA also has a scaling challenge. Currently, anyone
who has the ability to push data to ELPA for a package also has the
ability to push to any directory (package), not just the package they
are maintainer for. This means that access to write permission for the
ELPA repository needs to be restricted to trusted users, which in turn
places more pressure on those trusted users to manage requests to update
data. As the number of ELPA packages increases, either more people will
need to be trusted or more pressure placed on those who are. As you
increase the number of people with write access you also increase the
risks.


>
> MELPA cannot know possibly who is behind authors who offer those
> packages for distribution and who has access or who may do something
> malicious.
>

The situation with ELPA is not much better. Yes, the authors are
required to sign over copyright, but what does that really tell you
about the author. How much vetting is done to verify those copyright
assignments? How much vetting is done to verify the identities of those
people? More importantly, how much of the code is formally reviewed?

The assumption that because a package is from ELPA it is safe is wrong.
This is the danger - an expectation that because the package is from
ELPA it is more trustworthy than a package from MELPA. The only thing
you really know for certain about a package from ELPA is that it has a
GPL license and it does not recommend or require non-free software. Any
review of the code in the package is informal and not guaranteed to
occur after every update. The requirement to assign copyright and the
fact at least some informal review is performed does provide some level
of assurance you don't get from MELPa, but it is a mistake to assume
that just because a package comes from ELPA it is safe or does not
include any significant security issues.

> Some new similar package like angry-police-captain could serve for
> potential attacks.
>
> #+TITLE: <2020-10-23 Fri 18:28>  WTF angry-police-captain
> #+AUTHOR: Jean Louis
> - This should scrap information from a third party unknown website and
>   show it in minibuffer. Function does not work, and yes, it is just
>   one function inside. Good example of nonsensical
>   "packages". *Deleted*
>
> While similar packages can be made for entertainment they can be also
> used to track users and save data that should not be saved. Update to
> this package would not be checked by MELPA, and users who have enabled
> it would continue using it. And package could suddenly start doing
> something else. Author of the package could know how many users are
> using it as package is actually fetching from their website. By
> fetching the information from website the website can know many things
> about those users such as their locations, operating system and
> versions, etc. and could invoke specific malicious stuff for those
> specific users including send different information to users by their
> different location or other attributes.
>

and the same thing is possible with ELPA. You may need to be a little
mor subtle and you may need to play 'the long game', but there is
nothing inherent in the ELPA process which provides protection against
this other than the valuable and incredible dedication of Emacs
developers. The problem is, informal processes for code review are
notoriously unreliable. We just have to look at what happened in the
openssl project to see how badly things can go wrong despite all good
intentions. In fact, your own words demonstrate the issue. You have a
belief/expectation that ELPA packages are safer than MELPA packages
because they are from a source you trust. However, without any formal
review of the code in those packages, that level of trust may not be
warranted.

So how big a risk are ELPA packages in reality? This is a difficult
thing to quantify. Yes, I do think there is lower risk with ELPA than
MELPA because even informal review is better than no review. I also
think that if you wanted to introduce a malicious package into the Emacs
ecosystem you would follow the path of least resistance, which is MELPA.
I also think the risk and reward calculations make Emacs packages a
lower risk - the effort required to get a malicious package adopted and
the rewards such effort would provide just don't add up. There are far
more rewarding options out there. However, I do think people need to
install packages with caution, regardless of whether they are from ELPA
or MELPA or anywhere else.

> For that reason MELPA's automatic pulling of packages and race to
> offer "large package repository" is rather by its design detrimental
> for future. I hope it will change, but currently that is unlikely.
>

The automatic pulling is not the issue. As long as there is no formal
review of code in packages, any method used is vulnerable. Regardless of
the approach, at the end of the day your trusting the author/maintainer
will do the right thing. This is true for both MELPA and ELPA. It also
includes trusting the maintainer won't make a mistake and that they have
good operational security and are not themselves compromised.


<snip>

>> So, like MELPA, all you really have to go on is package
>> reputation. You cannot have any high level of confidence a package
>> does not contain malicious code other than an expectation that if it
>> is used by a sufficiently large enough number of users, it is
>> unlikely.
>
> Interpreting statistics is not for everyone. It would be nice that
> users give a human feedback which can be used for package reputation.
>
> If one counts "download statistics" that is incorrect to be used for
> reputation.
>
> If let us say imaginary, a package about angry-police-captain would
> contain some malicious code, then if user cannot differentiate what is
> reputation and download, then number of 1000+ downloads would be quite
> convincing to load the package.
>
> Download number is now used for reputation as it is currently the only
> attribute that may be obtained.
>

that is not what I meant by reputation and number of downloads is not
the metric I use. What I meant by reputation is what others write about
the package - what is discussed in forums, what comes up with a google
search etc. I maintain a JS package which has over 100k downloads per
week. This means nothing to me and I suspect to most users. The
reputation for the package is based on what is in the issue/bug tracker,
how quickly issues are addressed and how many other packages or systems
use the package.

I think we have exhausted this topic now. It really is something which
should be discussed on the emacs-devel list rather than an org list. I
do think people probably need to be more aware of the risks associated
with all emacs packages, regardless of source, but probably best in a
more general emacs forum rather than this list.

--
Tim Cross