problems cloning using the http protocol

problems cloning using the http protocol

[Dias Badekas]

At work being behind a corporate firewall I have no choice but to use
the http protocol inorder to get the latest development of orgmode.

unfortunately it doesn't seem to work.
Here's what I get:

bash> export http_proxy=http://user:pass@proxyhostname:80/
bash> git clone http://repo.or.cz/r/org-mode.git
      Initialized empty Git repository in /home/dias/Mystuff/Emacs/downloads/org-mode/.git/
      Cannot get remote repository information.
      Perhaps git-update-server-info needs to be run there?

Can this be fixed ?

Re: problems cloning using the http protocol

[Scot Becker]

Just tried:
   git clone http://repo.or.cz/r/org-mode.git

behind my corporate firewall, and it worked.  It's slow, however.  So
I don't think the problem is on the server end.  Did you try any other
projects (using git+http proxy)?

I don't suppose anyone knows how to git org develpment releases
sometimes behind an http proxy and sometimes directly (as on a laptop
that is sometimes behind a firewall, and sometimes not.)  I couldn't
make this to work.

Scot

Re: problems cloning using the http protocol

[Nick Dokos]

Scot Becker <scot.becker@gmail.com> wrote:

> I don't suppose anyone knows how to git org develpment releases
> sometimes behind an http proxy and sometimes directly (as on a laptop
> that is sometimes behind a firewall, and sometimes not.)  I couldn't
> make this to work.
> 

AFAIK, with just an http proxy, you are stuck with http.

If your company provides a SOCKS proxy (a big if, I know), then you can
forego http and always use git. You'd need to set up a SOCKS client
(dante or tsocks), do a little configuration and then if you are behind
the firewall, you'd do e.g. ``socksify git pull'', rather than ``git pull''.

HTH,
Nick

Re: problems cloning using the http protocol

[Scot Becker]

thanks, Nick

On Tue, Jul 14, 2009 at 2:26 PM, Nick Dokos<nicholas.dokos@hp.com> wrote:
> Scot Becker <scot.becker@gmail.com> wrote:
>
>> I don't suppose anyone knows how to git org develpment releases
>> sometimes behind an http proxy and sometimes directly (as on a laptop
>> that is sometimes behind a firewall, and sometimes not.)  I couldn't
>> make this to work.
>>
>
> AFAIK, with just an http proxy, you are stuck with http.
>
> If your company provides a SOCKS proxy (a big if, I know), then you can
> forego http and always use git. You'd need to set up a SOCKS client
> (dante or tsocks), do a little configuration and then if you are behind
> the firewall, you'd do e.g. ``socksify git pull'', rather than ``git pull''.
>
> HTH,
> Nick
>

Re: problems cloning using the http protocol

[Bernt Hansen]

Scot Becker <scot.becker@gmail.com> writes:

> Just tried:
>    git clone http://repo.or.cz/r/org-mode.git
>
> behind my corporate firewall, and it worked.  It's slow, however.

HTTP will always be slower than the git protocol.  HTTP is dumb so there
is no way to transfer only part of a pack that contains the objects you
are missing.  You'll get the entire pack file each time via the HTTP
protocol.

> So I don't think the problem is on the server end.  Did you try any
> other projects (using git+http proxy)?
>
> I don't suppose anyone knows how to git org develpment releases
> sometimes behind an http proxy and sometimes directly (as on a laptop
> that is sometimes behind a firewall, and sometimes not.)  I couldn't
> make this to work.

There are multiple options
------------------------------------------------------------------------
UNTESTED

You might be able to get away with creating a second remote for the
project like this:

[remote "origin"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = http://repo.or.cz/r/org-mode.git

[remote "origin2"]
	url = git://repo.or.cz/org-mode.git
	fetch = +refs/heads/*:refs/remotes/origin/*

then

  git fetch origin
  uses the HTTP protocol

and

  git fetch origin2
  uses the git protocol

You can set up origin to be whichever you use more often (or switch as
required.)  The only differences between these remotes are:

  - name of the remote - you can name it anything you like
  - the fetch = lines are identical - using the same remote ref
    'refs/remotes/origin'.  This should work okay since the repos are
    physically the same
  - the url specifies the transport method to use.

------------------------------------------------------------------------

Just use two remotes

origin for your fast git protocol access
http for your slow http protocol access

then you'll have remotes like origin/master and http/master

If you can't git pull origin, you can manually git fetch http and git
merge http/master or whatever is appropriate

------------------------------------------------------------------------

I carry my org-mode.git around on my USB stick -- so I can just
clone/pull/fetch from that without internet access.  I'll update my
workstation/laptop repo when it has net access and then push to the usb
stick.

HTH,
Bernt

Re: problems cloning using the http protocol

[Scot Becker]

Bernt,

> HTTP will always be slower than the git protocol.  HTTP is dumb so there
> is no way to transfer only part of a pack that contains the objects you
> are missing.  You'll get the entire pack file each time via the HTTP
> protocol.

Ah, that explains it.  That makes the benefit of git via http vs. the
devel.tar.gz somewhat less.


Thanks for your explanations.  That helps.  Git is a wide wide world
of which I have only seen a sliver.

Scot

Re: problems cloning using the http protocol

[Dias Badekas]

If this is relevant your question, you may want to look into setting up
a http CONNECT proxy, to use the efficient git protocol through your
corporate proxy.
I found these articles while searching for my problem.

http://www.emilsit.net/blog/archives/how-to-use-the-git-protocol-through-a-http-connect-proxy/
http://limlian.blogspot.com/2008/08/use-http-proxy-for-git-client.html


Bernt Hansen <bernt@norang.ca> writes:

> Scot Becker <scot.becker@gmail.com> writes:
>
>
>> So I don't think the problem is on the server end.  Did you try any
>> other projects (using git+http proxy)?
>>
>> I don't suppose anyone knows how to git org develpment releases
>> sometimes behind an http proxy and sometimes directly (as on a laptop
>> that is sometimes behind a firewall, and sometimes not.)  I couldn't
>> make this to work.
>
> There are multiple options
> ------------------------------------------------------------------------
> UNTESTED
>
> You might be able to get away with creating a second remote for the
> project like this:
>
> [remote "origin"]
> 	fetch = +refs/heads/*:refs/remotes/origin/*
> 	url = http://repo.or.cz/r/org-mode.git
>
> [remote "origin2"]
> 	url = git://repo.or.cz/org-mode.git
> 	fetch = +refs/heads/*:refs/remotes/origin/*
>
> then
>
>   git fetch origin
>   uses the HTTP protocol
>
> and
>
>   git fetch origin2
>   uses the git protocol
>
> You can set up origin to be whichever you use more often (or switch as
> required.)  The only differences between these remotes are:
>
>   - name of the remote - you can name it anything you like
>   - the fetch = lines are identical - using the same remote ref
>     'refs/remotes/origin'.  This should work okay since the repos are
>     physically the same
>   - the url specifies the transport method to use.
>
> ------------------------------------------------------------------------
>
> Just use two remotes
>
> origin for your fast git protocol access
> http for your slow http protocol access
>
> then you'll have remotes like origin/master and http/master
>
> If you can't git pull origin, you can manually git fetch http and git
> merge http/master or whatever is appropriate
>
> ------------------------------------------------------------------------
>
> I carry my org-mode.git around on my USB stick -- so I can just
> clone/pull/fetch from that without internet access.  I'll update my
> workstation/laptop repo when it has net access and then push to the usb
> stick.
>
> HTH,
> Bernt

Re: problems cloning using the http protocol

[Dias Badekas]

Thanks for all your help. 
Turns out that the git version was 1.4.* and has a bug that prevented
proper use of http_proxy settings.

Upgrading to git 1.6.3.3 solved the problem.

Now I can use git clone http://.... 


Scot Becker <scot.becker@gmail.com> writes:

> Just tried:
>    git clone http://repo.or.cz/r/org-mode.git
>
> behind my corporate firewall, and it worked.  It's slow, however.  So
> I don't think the problem is on the server end.  Did you try any other
> projects (using git+http proxy)?
>
> I don't suppose anyone knows how to git org develpment releases
> sometimes behind an http proxy and sometimes directly (as on a laptop
> that is sometimes behind a firewall, and sometimes not.)  I couldn't
> make this to work.
>
> Scot

Re: problems cloning using the http protocol

[Giovanni Ridolfi]

--- Mar 14/7/09, Dias Badekas <dbadekas@aia.gr> ha scritto:
> At work being behind a corporate firewall I have no choice
> but to use
> the http protocol inorder to get the latest development of
> orgmode.

why don't you download the zip file?

http://orgmode.org/index.html#sec-3.2

"For people who cannot use git, we provide zip 
or tar.gz snapshot release files corresponding 
to the latest git version. "

cheers,
Giovanni 

Re: problems cloning using the http protocol

[Richard Riley]

Giovanni Ridolfi <giovanni.ridolfi@yahoo.it> writes:

> --- Mar 14/7/09, Dias Badekas <dbadekas@aia.gr> ha scritto:
>> At work being behind a corporate firewall I have no choice
>> but to use
>> the http protocol inorder to get the latest development of
>> orgmode.
>
> why don't you download the zip file?
>
> http://orgmode.org/index.html#sec-3.2
>
> "For people who cannot use git, we provide zip 
> or tar.gz snapshot release files corresponding 
> to the latest git version. "
>
> cheers,
> Giovanni 

I would guess that it's not as up to date (or is it dynamically
generated on request?) and its not possible to simply "git pull" the
latest which is a great boon.

Re: problems cloning using the http protocol

[Giovanni Ridolfi]

--- Mar 14/7/09, Richard Riley <rileyrgdev@googlemail.com> ha scritto:
> Giovanni Ridolfi <giovanni.ridolfi@yahoo.it>
> writes:
> 
> > --- Mar 14/7/09, Dias Badekas <dbadekas@aia.gr>
> ha scritto:
> >> At work being behind a corporate firewall I have
> no choice
> >> but to use
> >> the http protocol inorder to get the latest
> development of
> >> orgmode.
> >
> > why don't you download the zip file?
> >
> > http://orgmode.org/index.html#sec-3.2
> >
> > "For people who cannot use git, we provide zip 
> > or tar.gz snapshot release files corresponding 
> > to the latest git version. "
> 
> I would guess that it's not as up to date 
        ^^^^^^^^^
????????  ?-|
Errm, Richard, please !! Before 'guessing', please, 
search and verify !! ;-)

> (or is it dynamically generated on request?) 
http://article.gmane.org/gmane.emacs.orgmode/10688

I think that words are important. If Carsten writes:

"zip [...] snapshot release files *corresponding*
 to the *latest git version*."

he means "corresponding to the latest git version",
or he's a liar. Is he?

cheers,

Giovanni

Re: problems cloning using the http protocol

[Bastien]

>> "For people who cannot use git, we provide zip 
>> or tar.gz snapshot release files corresponding 
>> to the latest git version. "

Actually this cannot be true.  AFAIK the snapshots were
done by Carsten, not by the server.  I fixed that so now
the snapshots are updated each hour and will be as much
"close" as possible to the git repo.

  http://orgmode.org/org-latest.zip
  http://orgmode.org/org-latest.tar.gz

(Old org-snapshot.* still available.)

But of course, it's less overhead to pull from git when
possible.

-- 
 Bastien

Re: problems cloning using the http protocol

[Dias Badekas]

I was under the impression that downloading/pulling only the changes
from git (even with the *dumb* http method) would be "better" than
getting repeatedly the whole archive.

According to some posts on this thread, maybe the above assumption is
not correct with git. is that right ?

Still, it seems simpler to just do a "git pull" instead of going thru a
download, unpack cycle



Giovanni Ridolfi <giovanni.ridolfi@yahoo.it> writes:

> why don't you download the zip file?
>
> http://orgmode.org/index.html#sec-3.2
>
> "For people who cannot use git, we provide zip 
> or tar.gz snapshot release files corresponding 
> to the latest git version. "
>
> cheers,
> Giovanni 
>
>
>