Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Daily Show (Comedy Central) cannot be downloaded anymore #11600

Closed
hstracker90 opened this issue Jan 4, 2017 · 14 comments
Closed

The Daily Show (Comedy Central) cannot be downloaded anymore #11600

hstracker90 opened this issue Jan 4, 2017 · 14 comments

Comments

@hstracker90
Copy link

With the new year they changed something on the website. The different formats are shown, but the stream cannot be found (HTTP Error 404). I can stream the program on the website with no problem. Please look into this. As always, thank you very much!

PS C:\WINDOWS\system32> ytdl -f http-2128 http://www.cc.com/full-episodes/1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040 --verbose
[debug] System config: []
[debug] User config: ['-o', 'C:/Users/hstracker/Downloads/%(title)s.%(ext)s']
[debug] Custom config: []
[debug] Command-line args: ['-f', 'http-2128', 'http://www.cc.com/full-episodes/1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2017.01.02
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg 2.8.git, ffprobe 2.8.git, rtmpdump 2.4
[debug] Proxy map: {}
[ComedyCentralFullEpisodes] 1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040: Downloading webpage
[ComedyCentralFullEpisodes] 1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040: Downloading JSON metadata
[ComedyCentralFullEpisodes] 2d277158-b85a-11e6-bfd4-0026b9414f30: Downloading info
[ComedyCentralFullEpisodes] 551b4af3-73f1-4683-9fcd-4f3941334224: Extracting information
[ComedyCentralFullEpisodes] 551b4af3-73f1-4683-9fcd-4f3941334224: Downloading video urls
[ComedyCentralFullEpisodes] af4a578d-e2fa-4ccf-93aa-b80c67fcdb4e: Extracting information
[ComedyCentralFullEpisodes] af4a578d-e2fa-4ccf-93aa-b80c67fcdb4e: Downloading video urls
[ComedyCentralFullEpisodes] eafbee67-1e22-46bf-80af-4fe957b3d4ce: Extracting information
[ComedyCentralFullEpisodes] eafbee67-1e22-46bf-80af-4fe957b3d4ce: Downloading video urls
[ComedyCentralFullEpisodes] b990d987-1867-428e-a90e-e7ba82475b4d: Extracting information
[ComedyCentralFullEpisodes] b990d987-1867-428e-a90e-e7ba82475b4d: Downloading video urls
[download] Downloading playlist: January 3, 2017 - Michael Che
[ComedyCentralFullEpisodes] playlist January 3, 2017 - Michael Che: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
[debug] Invoking downloader on 'http://viacommtvstrmfs.fplive.net/gsp.comedystor/com/dailyshow/TDS/season_22/040/ds_22_0
40_act1_ngzsdi8x7z_960x540_2128.mp4'
ERROR: unable to download video data: HTTP Error 404: Not Found
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0hst59yw\build\youtube_dl\YoutubeDL.py", line 1699, in process_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0hst59yw\build\youtube_dl\YoutubeDL.py", line 1641, in dl
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0hst59yw\build\youtube_dl\downloader\common.py", line 353, in download
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0hst59yw\build\youtube_dl\downloader\http.py", line 61, in real_download
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0hst59yw\build\youtube_dl\YoutubeDL.py", line 2001, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default
urllib.error.HTTPError: HTTP Error 404: Not Found

@gwemmie
Copy link

gwemmie commented Jan 4, 2017

I'm getting a slightly different error (but am on Linux instead of Windows):

$ youtube-dl "http://www.cc.com/full-episodes/1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040" --verbose
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['http://www.cc.com/full-episodes/1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.12.01
[debug] Python version 3.5.2 - Linux-4.8.11-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 3.2.1, ffprobe 3.2.1, rtmpdump 2.4
[debug] Proxy map: {}
[ComedyCentralFullEpisodes] 1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040: Downloading webpage
[ComedyCentralFullEpisodes] 1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040: Downloading JSON metadata
[ComedyCentralFullEpisodes] 2d277158-b85a-11e6-bfd4-0026b9414f30: Downloading info
[ComedyCentralFullEpisodes] 551b4af3-73f1-4683-9fcd-4f3941334224: Extracting information
[ComedyCentralFullEpisodes] 551b4af3-73f1-4683-9fcd-4f3941334224: Downloading video urls
[ComedyCentralFullEpisodes] af4a578d-e2fa-4ccf-93aa-b80c67fcdb4e: Extracting information
[ComedyCentralFullEpisodes] af4a578d-e2fa-4ccf-93aa-b80c67fcdb4e: Downloading video urls
[ComedyCentralFullEpisodes] eafbee67-1e22-46bf-80af-4fe957b3d4ce: Extracting information
[ComedyCentralFullEpisodes] eafbee67-1e22-46bf-80af-4fe957b3d4ce: Downloading video urls
[ComedyCentralFullEpisodes] b990d987-1867-428e-a90e-e7ba82475b4d: Extracting information
[ComedyCentralFullEpisodes] b990d987-1867-428e-a90e-e7ba82475b4d: Downloading video urls
[download] Downloading playlist: January 3, 2017 - Michael Che
[ComedyCentralFullEpisodes] playlist January 3, 2017 - Michael Che: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
[debug] Invoking downloader on 'http://viacommtvstrmfs.fplive.net/gsp.comedystor/com/dailyshow/TDS/season_22/040/ds_22_040_act1_ngzsdi8x7z_1920x1080_5128.mp4'
ERROR: unable to download video data: <urlopen error [Errno 110] Connection timed out>
Traceback (most recent call last):
  File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/lib/python3.5/http/client.py", line 1106, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1102, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/lib/python3.5/http/client.py", line 849, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.5/socket.py", line 711, in create_connection
    raise err
  File "/usr/lib/python3.5/socket.py", line 702, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/lib/python3.5/site-packages/youtube_dl/downloader/common.py", line 353, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python3.5/site-packages/youtube_dl/downloader/http.py", line 61, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 466, in open
    response = self._open(req, data)
  File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
    '_open', req)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/site-packages/youtube_dl/utils.py", line 872, in http_open
    req)
  File "/usr/lib/python3.5/urllib/request.py", line 1256, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 110] Connection timed out>

@phihag
Copy link
Contributor

phihag commented Jan 4, 2017

I'm seeing both of these errors; the 404 from Germany, the Connection timed out from the US.

@StevenDTX
Copy link

StevenDTX commented Jan 4, 2017

I think this is closely related to #11148

@sander1
Copy link
Contributor

sander1 commented Jan 4, 2017

I think this is closely related to #11148

Doesn't look like it. #11148 is related to finding the mgid value. This issue is about missing files and unreachable hosts.

@GravisZro
Copy link

I'm in the US and also getting a timeout. However, the video works on the page using the Flash Player. Perhaps it's expecting a particular HTTP header field values to prevent things like youtube-dl.

$ youtube-dl "http://www.cc.com/full-episodes/1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040" --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'http://www.cc.com/full-episodes/1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040', u'--verbose']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.01.05
[debug] Python version 2.7.13 - Linux-4.7.0-1-amd64-x86_64-with-Ubuntu-12.04-precise
[debug] exe versions: none
[debug] Proxy map: {}
[ComedyCentralFullEpisodes] 1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040: Downloading webpage
[ComedyCentralFullEpisodes] 1wb7rt/the-daily-show-with-trevor-noah-january-3--2017---michael-che-season-22-ep-22040: Downloading JSON metadata
[ComedyCentralFullEpisodes] 2d277158-b85a-11e6-bfd4-0026b9414f30: Downloading info
[ComedyCentralFullEpisodes] 551b4af3-73f1-4683-9fcd-4f3941334224: Extracting information
[ComedyCentralFullEpisodes] 551b4af3-73f1-4683-9fcd-4f3941334224: Downloading video urls
[ComedyCentralFullEpisodes] af4a578d-e2fa-4ccf-93aa-b80c67fcdb4e: Extracting information
[ComedyCentralFullEpisodes] af4a578d-e2fa-4ccf-93aa-b80c67fcdb4e: Downloading video urls
[ComedyCentralFullEpisodes] eafbee67-1e22-46bf-80af-4fe957b3d4ce: Extracting information
[ComedyCentralFullEpisodes] eafbee67-1e22-46bf-80af-4fe957b3d4ce: Downloading video urls
[ComedyCentralFullEpisodes] b990d987-1867-428e-a90e-e7ba82475b4d: Extracting information
[ComedyCentralFullEpisodes] b990d987-1867-428e-a90e-e7ba82475b4d: Downloading video urls
[download] Downloading playlist: January 3, 2017 - Michael Che
[ComedyCentralFullEpisodes] playlist January 3, 2017 - Michael Che: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
[debug] Invoking downloader on u'http://viacommtvstrmfs.fplive.net/gsp.comedystor/com/dailyshow/TDS/season_22/040/ds_22_040_act1_ngzsdi8x7z_1920x1080_5128.mp4'
ERROR: unable to download video data: <urlopen error [Errno 110] Connection timed out>
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 353, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 61, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 429, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 447, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 877, in http_open
    req)
  File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno 110] Connection timed out>

@phihag
Copy link
Contributor

phihag commented Jan 5, 2017

We may be able to use HLS URLs instead of the hidden HTTP ones (which were basically guessed from the RTMP URLs).

@phihag phihag closed this as completed in dbaf601 Jan 5, 2017
@phihag
Copy link
Contributor

phihag commented Jan 5, 2017

I've implemented it as such. Using m3u playlists is by no means optimal, but it's better than nothing for now. With the change, youtube-dl :tds works for me from Germany.

@japanesephundroid
Copy link

@yan12125
Copy link
Collaborator

yan12125 commented Jan 9, 2017

@japanesephundroid Maybe the same issue as #11624

@japanesephundroid
Copy link

@yan12125 Yes its the same issue both try to use the hidden http link (the issue @phihag fixed 4 days ago) instead of using HLS.

@andlabs
Copy link

andlabs commented Jan 9, 2017

I'm not sure if this is the same issue either or not, but it seems that certain old full-episodes links redirect to episodes, so they can't be downloaded. Example:

$ youtube-dl --verbose 'http://www.cc.com/full-episodes/wu5xuj/the-daily-show-with-jon-stewart-november-8--2011---bill-clinton-season-17-ep-17018'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'http://www.cc.com/full-episodes/wu5xuj/the-daily-show-with-jon-stewart-november-8--2011---bill-clinton-season-17-ep-17018']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.01.08
[debug] Python version 2.7.10 - Darwin-15.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.1.2, ffprobe 3.1.2, rtmpdump 2.4
[debug] Proxy map: {}
[ComedyCentralFullEpisodes] wu5xuj/the-daily-show-with-jon-stewart-november-8--2011---bill-clinton-season-17-ep-17018: Downloading webpage
[ComedyCentralFullEpisodes] wu5xuj/the-daily-show-with-jon-stewart-november-8--2011---bill-clinton-season-17-ep-17018: Downloading JSON metadata
ERROR: An extractor error has occurred. (caused by KeyError(u'data',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 359, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/comedycentral.py", line 58, in _real_extract
    mgid = feed['result']['data']['id']
KeyError: u'data'
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 694, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 365, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e)
ExtractorError: An extractor error has occurred. (caused by KeyError(u'data',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

If you open the link in a browser it will turn full-episodes into just episodes, which won't work either:

$ youtube-dl --verbose 'http://www.cc.com/episodes/wu5xuj/the-daily-show-with-jon-stewart-november-8--2011---bill-clinton-season-17-ep-17018'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'http://www.cc.com/episodes/wu5xuj/the-daily-show-with-jon-stewart-november-8--2011---bill-clinton-season-17-ep-17018']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.01.08
[debug] Python version 2.7.10 - Darwin-15.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.1.2, ffprobe 3.1.2, rtmpdump 2.4
[debug] Proxy map: {}
[ComedyCentral] the-daily-show-with-jon-stewart-november-8--2011---bill-clinton-season-17-ep-17018: Downloading webpage
ERROR: Unable to extract mgid; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/mtv.py", line 218, in _extract_mgid
    og_url = self._og_search_video_url(webpage)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 780, in _og_search_video_url
    return self._html_search_regex(regexes, html, name, **kargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 671, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 662, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract video url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 694, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 359, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/mtv.py", line 240, in _real_extract
    mgid = self._extract_mgid(webpage)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/mtv.py", line 234, in _extract_mgid
    r'embed/(mgid:.+?)["\'&?/]', sm4_embed, 'mgid', default=default)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 662, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract mgid; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

This is with 2017.01.08. If this is a different issue I'll split it.

@japanesephundroid
Copy link

japanesephundroid commented Jan 10, 2017

@andlabs

I'm not sure if this is the same issue either or not, but it seems that certain old full-episodes links redirect to episodes, so they can't be downloaded

Comedy Central only keeps a limited number of full episodes available, most are just removed from the site and some can be only played with a TV subscription. The John Stewart you tried is not available on the site for viewing in the full episode format.

If this is a different issue I'll split it.

As far as I can tell this is not an issue, this is intended behavior, there are no videos to download.

@andlabs
Copy link

andlabs commented Jan 10, 2017

Did they stop making the episodes available as individual clips that combine to form the full episodes? Because I remember that's how it worked a few years ago.

@japanesephundroid
Copy link

japanesephundroid commented Jan 10, 2017

@andlabs

Did they stop making the episodes available as individual clips that combine to form the full episodes? Because I remember that's how it worked a few years ago.

Those are still available, every clip is a separate web page. That link you posted should not work, but Comedy Central still has some links from the episode there for backwards compatibility.

If you can find the first one the next one is usually available as a suggestion at the end, or it was the last time I tried to watch an episode this way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants