Can't access remote repository if the URL contains Chinese word #945

YueLinHo · 2016-11-06T16:08:17Z

[v] I was not able to find an open or closed issue matching what I'm seeing

Setup

Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

64 bit

$ git --version --build-options
git version 2.10.2.windows.1

Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver
Microsoft Windows [版本 6.1.7601]

Win7 64 Pro

What options did you set as part of the installation? Or did you choose the defaults?

> type "C:\Program Files\Git\etc\install-options.txt"
Path Option: Cmd
SSH Option: OpenSSH
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled

Any other interesting things about your environment that might be related to the issue you're seeing?

Suppose 2.10.1 has the same issue.

Study some, and compare the changes between 2.10.0 and 2.10.1, then I guess the changes of url.c in
commit introduce hex2chr() for converting two hexadecimal digits to a character may be the key?

Also see TortoiseGit issue 2859

Details

Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

CMD

What commands did you run to trigger this issue? If you can provide a Minimal, Complete, and Verifiable example this will help us understand the issue.

Can't clone if URL contains Chinese word(s)

$ git.exe clone --progress -v "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\Test中文2.10.2"

Or add a remote and its URL contains Chinese word(s). Then can't perform fetch/push anymore.

Note: English URL works fine.

What did you expect to occur after running these commands?

Like Git Bash does:

$ git.exe clone --progress -v "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\Test中文2.10.2"
Cloning into 'D:\Temp\i2859\Test中文2.10.2'...
done.

What actually happened instead?

D:\Temp\i2589>git.exe clone --progress -v "D:\Temp\i2589\Test中文.git" "D:\Temp\i2589\Test中文2.10.2"
Cloning into 'D:\Temp\i2589\Test中文2.10.2'...
"git-upload-pack 'D:\Temp\i2589\Testä¸æ��.git'": git-upload-pack 'D:\Temp\i2589\Testä¸æ��.git': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Warning: Your console font probably doesn't support Unicode. If you experience strange characters in the output, consider switching to a TrueType font such as Consolas!

Update 1

TortoiseGit interface shows those two strings normally:

And test with true type in cmd.exe:

D:\Temp\i2589>git clone "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\中文2.10.2"
Cloning into 'D:\Temp\i2859\中文2.10.2'...
"git-upload-pack 'D:\Temp\i2859\Testä¸æ��.git'": git-upload-pack 'D:\Temp\i2859
\Testä¸æ��.git': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

That warning is gone.

Update 2

Other testing results: (TortoiseGit calls git.exe)

As you can see:

2.10.0 is last good for TortoiseGit
2.10.1 is first bad. Bash works fine, but CMD. (based on that TortoiseGit calls git.exe, and the result in my last comment)

Workaround

call set LC_ALL=C in cmd.exe

The text was updated successfully, but these errors were encountered:

PhilipOakley · 2016-11-06T16:46:02Z

----- Original Message -----
From: Yue Lin Ho
To: git-for-windows/git
Sent: Sunday, November 06, 2016 4:08 PM
Subject: [git-for-windows/git] Can't access remote repository if the URL contains Chinese word (#945)

a.. [v] I was not able to find an open or closed issue matching what I'm seeing

Setup
a.. Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options
git version 2.10.2.windows.1
a.. Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver
Microsoft Windows [版本 6.1.7601]
a.. What options did you set as part of the installation? Or did you choose the
defaults?

One of the following:

type "C:\Program Files\Git\etc\install-options.txt"
type "C:\Program Files (x86)\Git\etc\install-options.txt"
type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt
Path Option: Cmd
SSH Option: OpenSSH
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
a.. Any other interesting things about your environment that might be related
to the issue you're seeing?
Suppose 2.10.1 has the same issue.

Study some, and compare the changes between 2.10.0 and 2.10.1, then I guess the changes of url.c in
commit introduce hex2chr() for converting two hexadecimal digits to a character may be the key?

Also see TortoiseGit issue 2859

Details
a.. Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other
CMD

a.. What commands did you run to trigger this issue? If you can provide a
Minimal, Complete, and Verifiable example
this will help us understand the issue.

Add a remote and its URL contains Chinese word(s). Then can't perform clone/fetch/push anymore.

Note: English URL works fine.

a.. What did you expect to occur after running these commands?

Like Git Bash does:

$ git.exe clone --progress -v "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\Test中文2.10.2"
Cloning into 'D:\Temp\i2859\Test中文2.10.2'...
done.
a.. What actually happened instead?
D:\Temp\i2589>git.exe clone --progress -v "D:\Temp\i2589\Test中文.git" "D:\Temp\i2589\Test中文2.10.2"
Cloning into 'D:\Temp\i2589\Test中文2.10.2'...
"git-upload-pack 'D:\Temp\i2589\Testä¸æ��.git'": git-upload-pack 'D:\Temp\i2589\Testä¸æ��.git': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Warning: Your console font probably doesn't support Unicode. If you experience strange characters in the output, consider switching to a TrueType font such as Consolas!

hello,

I think this warning message "Your console font probably doesn't support Unicode. If you experience strange characters in the output, consider switching to a TrueType font such as Consolas!" Is at the heart of the problem.

It looks like an encoding problem. Git assumes you use UTF-8 unicode. I guess that is not how your chinese characters/words are being encoded when passed to git, so while they are displayed correctly on screen, they are different 8-bit byte sequences.

Try the suggestion of "switching to a TrueType font such as Consolas" and seeing if that has an effect.

Philip

YueLinHo · 2016-11-07T02:20:37Z

@PhilipOakley

Hi,

I test it again with true type:

D:\Temp\i2589>git clone "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\中文2.10.2"
Cloning into 'D:\Temp\i2859\中文2.10.2'...
"git-upload-pack 'D:\Temp\i2859\Testä¸æ��.git'": git-upload-pack 'D:\Temp\i2859
\Testä¸æ��.git': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

That warning is gone.

Note: Git Bash works fine, but Windows Command Prompt.

YueLinHo · 2016-11-07T09:35:27Z

Post earlier testing results: (TortoiseGit calls git.exe)

As you can see:

2.10.0 is last good for TortoiseGit
2.10.1 is first bad. Bash works fine, but CMD. (based on that TortoiseGit calls git.exe, and the result in my last comment)

So, I compared the code between tag v2.10.0.windows.1 (commit 1f448f5) and v2.10.1.windows.1 (commit 3374789), and my first guessing is the changes of url.c in commit introduce hex2chr() for converting two hexadecimal digits to a character. But, I am not sure it is the root cause.

dscho · 2016-11-07T13:10:43Z

Please call set LC_ALL=C in cmd.exe and try to clone again.

YueLinHo · 2016-11-07T14:12:41Z

@dscho OH~ YES~ set LC_ALL=C works. (Oh~ Happy~~ ^____^)

Thank you!!!!!

PhilipOakley · 2016-11-07T14:19:09Z

Hi,

I still suspect that it's an encoding issue.

Your results show that this is not a problem for GfW 2.10.1/2 from git bash, so we are doing something right.

The results then show that when TortoiseGit does the command then it fails. So it will be somewhere in the tortoiseGit interface. (with the same suspect that it is a 'bad' code page / encoding issue).

Can you capture the two strings as byte sequences to see if they follow the UTF-8 and code page encodings?

P.
----- Original Message -----
From: Yue Lin Ho
To: git-for-windows/git
Cc: Philip Oakley ; Mention
Sent: Monday, November 07, 2016 9:35 AM
Subject: Re: [git-for-windows/git] Can't access remote repository if the URL contains Chinese word (#945)

Post earlier testing result: (TortoiseGit calls git.exe)

As you can see:

2.10.0 is last good for TortoiseGit
Bash works fine, but CMD (based on that TortoiseGit calls git.exe, and the result in my last comment).

So, I compared the code between tag v2.10.0.windows.1 (commit 1f448f5) and v2.10.1.windows.1 (commit 3374789), and my first guessing is the changes of url.c in commit introduce hex2chr() for converting two hexadecimal digits to a character. But, I am not sure it is the root cause.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

YueLinHo · 2016-11-07T14:36:42Z

@PhilipOakley

I still suspect that it's an encoding issue.
Your results show that this is not a problem for GfW 2.10.1/2 from git bash, so we are doing something right.
The results then show that when TortoiseGit does the command then it fails. So it will be somewhere in the tortoiseGit interface. (with the same suspect that it is a 'bad' code page / encoding issue).

Can you capture the two strings as byte sequences to see if they follow the UTF-8 and code page encodings?

TortoiseGit interface shows those two strings normally:

So, I guess it follows. :-)

YueLinHo · 2016-11-09T13:47:57Z

I can build Git for Windows and its installer now. Then, I tested the installer which revert the change by the commit I suspected, say commit d233097. The result is that clone still does not work. So, I think that that commit can be excluded. :-)

YueLinHo · 2016-11-09T23:36:48Z

I know Git for Windows can be built with VS2015 since 2.10.2. So, my next step may be debug it with VS2015? right?

YueLinHo · 2016-11-10T09:16:01Z

I built 2.10.0 and made a installer, then can't clone. XD

Looks like it's not source code problem. Any idea?

PhilipOakley · 2016-11-10T20:58:06Z

I build 2.10.0 and make a installer, then can't clone. XD
Looks like it's not source code problem. Any idea?

It maybe that your build completed, and overwrote your current install (within the sdk), and that there is some issue with your build.

Does your regular Git still work.

dscho · 2016-11-16T17:34:29Z

@YueLinHo is there a chance to teach TortoiseGit to set LC_ALL=C? Or would it help if Git for Windows' cmd\git.exe would set LC_ALL unless it was set already?

YueLinHo · 2016-11-17T12:16:54Z

@dscho

is there a chance to teach TortoiseGit to set LC_ALL=C?

I am not sure. I guess TortoiseGit can do so. I will find time to try it. (@csware Can TortoiseGit do it?)
For me, I added a env. var. LC_ALL=C, so that I can use TortoiseGit in daily work.

Or would it help if Git for Windows' cmd\git.exe would set LC_ALL unless it was set already?

I think it would help. ^_^

csware · 2016-11-17T12:40:46Z

TortoiseGit could set LC_ALL=C, however, then all localization is disabled, right?

YueLinHo · 2016-11-17T13:34:32Z

TortoiseGit could set LC_ALL=C, however, then all localization is disabled, right?

Right. So, you remind me that I want to know why my GfW 2.10.0 installer does not work out.
Are there other changes beside source code changes? package changed?

The regression where Git would not interpret non-ASCII characters passed from a CMD window correctly [has been fixed](git-for-windows/git#945). Signed-off-by: Johannes Schindelin <[email protected]>

dscho · 2016-11-18T13:01:06Z

TortoiseGit could set LC_ALL=C, however, then all localization is disabled, right?

@csware obviously I meant "unless LC_ALL was already defined".

dscho · 2016-11-18T13:02:08Z

I want to know why my GfW 2.10.0 installer does not work out.

@YueLinHo Would you kindly open a new ticket with a concise description what you tried and what you expected and what you got instead?

YueLinHo · 2016-11-18T23:10:31Z

Would you kindly open a new ticket with a concise description what you tried and what you expected and what you got instead?

Sure. See #965. ^_^

YueLinHo · 2017-01-23T07:02:37Z

Found there is a similar issue, say #1036, So I remove my system env, var. LC_ALL=C and re-test some for 2.11.0.windows.3:

Screenshot

Result

Looks like the issue is not fixed.

CLI - One of the Error Message

D:\Temp\gfwi1036\test2>git version
git version 2.11.0.windows.3

D:\Temp\gfwi1036\test2>git remote -v
local_one       D:/Temp/gfwi1036/測試 (fetch)
local_one       D:/Temp/gfwi1036/測試 (push)
origin  D:\Temp\gfwi1036\測試 (fetch)
origin  D:\Temp\gfwi1036\測試 (push)
share_on_server //192.168.x.xxx/_bk2/測試 (fetch)
share_on_server //192.168.x.xxx/_bk2/測試 (push)

D:\Temp\gfwi1036\test2>git fetch origin -v
"git-upload-pack 'D:\Temp\gfwi1036\測試'": git-upload-pack 'D:\Temp\gfwi1036\測
試': No such file or directory
fatal: Could not read from remote repository.

Git Gui - One of the Error Message

"git-upload-pack 'D:\Temp\gfwi1036\æ¸¬è©¦'": git-upload-pack 'D:\Temp\gfwi1036\æ¸¬è©¦': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

TortoiseGit

TortoiseGit 2.3.4+ has the same fix, and it works for me. Here is the testing of 2.3.8:

git.exe fetch -v --progress "origin"

From D:\Temp\gfwi1036\測試
= [up to date]      master     -> origin/master

After adding that system env. var. back, everything works.

YueLinHo · 2017-01-23T07:24:51Z

Suppose this issue is not just for Chinese words, but also for non-ASCII characters.
(Just adding a comment which making this issue to be found easily.)

dscho added the bug label Nov 7, 2016

PhilipOakley mentioned this issue Nov 12, 2016

git-bash.exe -c problem with non-ASCII string #953

Closed

dscho closed this as completed in git-for-windows/MINGW-packages@1bee65e Nov 17, 2016

dscho self-assigned this Nov 17, 2016

dscho added this to the v2.11.0 milestone Nov 17, 2016

YueLinHo mentioned this issue Jan 24, 2017

Since version 2.10.1, can't push/pull/fetch using 'Git Gui' when the remote path contains non-ASCII characters #1036

Closed

PhilipOakley mentioned this issue Feb 8, 2017

2.11.1 breaks log output for non-ASCII characters (e.g. German Umlauts) #1053

Closed

PhilipOakley mentioned this issue Mar 9, 2017

Broken cyrillic output in cmd #1087

Closed

fourpastmidnight mentioned this issue Mar 22, 2017

Non-ASCII characters displayed incorrectly in Windows CMD when running 'git log' in version 2.12.1 #1100

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't access remote repository if the URL contains Chinese word #945

Can't access remote repository if the URL contains Chinese word #945

YueLinHo commented Nov 6, 2016 •

edited

Loading

PhilipOakley commented Nov 6, 2016

YueLinHo commented Nov 7, 2016

YueLinHo commented Nov 7, 2016 •

edited

Loading

dscho commented Nov 7, 2016

YueLinHo commented Nov 7, 2016 •

edited

Loading

PhilipOakley commented Nov 7, 2016

YueLinHo commented Nov 7, 2016

YueLinHo commented Nov 9, 2016 •

edited

Loading

YueLinHo commented Nov 9, 2016 •

edited

Loading

YueLinHo commented Nov 10, 2016 •

edited

Loading

PhilipOakley commented Nov 10, 2016

dscho commented Nov 16, 2016

YueLinHo commented Nov 17, 2016

csware commented Nov 17, 2016

YueLinHo commented Nov 17, 2016

dscho commented Nov 18, 2016

dscho commented Nov 18, 2016 •

edited

Loading

YueLinHo commented Nov 18, 2016

YueLinHo commented Jan 23, 2017 •

edited

Loading

YueLinHo commented Jan 23, 2017

Can't access remote repository if the URL contains Chinese word #945

Can't access remote repository if the URL contains Chinese word #945

Comments

YueLinHo commented Nov 6, 2016 • edited Loading

Setup

Details

Update 1

Update 2

Workaround

PhilipOakley commented Nov 6, 2016

One of the following:

YueLinHo commented Nov 7, 2016

YueLinHo commented Nov 7, 2016 • edited Loading

dscho commented Nov 7, 2016

YueLinHo commented Nov 7, 2016 • edited Loading

PhilipOakley commented Nov 7, 2016

YueLinHo commented Nov 7, 2016

YueLinHo commented Nov 9, 2016 • edited Loading

YueLinHo commented Nov 9, 2016 • edited Loading

YueLinHo commented Nov 10, 2016 • edited Loading

PhilipOakley commented Nov 10, 2016

dscho commented Nov 16, 2016

YueLinHo commented Nov 17, 2016

csware commented Nov 17, 2016

YueLinHo commented Nov 17, 2016

dscho commented Nov 18, 2016

dscho commented Nov 18, 2016 • edited Loading

YueLinHo commented Nov 18, 2016

YueLinHo commented Jan 23, 2017 • edited Loading

Screenshot

Result

CLI - One of the Error Message

Git Gui - One of the Error Message

TortoiseGit

YueLinHo commented Jan 23, 2017

YueLinHo commented Nov 6, 2016 •

edited

Loading

YueLinHo commented Nov 7, 2016 •

edited

Loading

YueLinHo commented Nov 7, 2016 •

edited

Loading

YueLinHo commented Nov 9, 2016 •

edited

Loading

YueLinHo commented Nov 9, 2016 •

edited

Loading

YueLinHo commented Nov 10, 2016 •

edited

Loading

dscho commented Nov 18, 2016 •

edited

Loading

YueLinHo commented Jan 23, 2017 •

edited

Loading