Fix typo

Add basic docker-compose cluster

CHANGELOG.md

@@ -1,7 +1,44 @@
-# 0.6.0 (2018-09-18)
-
-## Week 6: Filters and Thumbnails
-
+# 0.8.0 (2018-10-02)
+
+## Week 8: Mixes
+
+Hello again!
+
+Mixes have been added with [`20130db`](https://github.com/omarroth/invidious/20130db), which makes it easy to create a playlist of related content. See [#188](https://github.com/omarroth/invidious/issues/188) for more info on how they work. Currently, they return the first 50 videos rather than a continuous feed to avoid tracking by Google/YouTube, which I think is a good trade-off between usability and privacy, and I hope other folks agree. You can create mixes by adding `RD` to the beginning of a video ID, an example is provided [here](https://www.invidio.us/mix?list=RDYE7VzlLtp-4) based on Big Buck Bunny. I've been quite happy with the results returned for the mixes I've tried, and it is not limited to music, which I think is a big plus. To emulate a continuous feed provided many are used to, using the last video of each mix as a new 'seed' has worked well for me. In the coming week I'd like to to add playback support in the player to listen to these easily.
+
+A very big thanks to [**@flourgaz**](https://github.com/flourgaz) for Docker support with [#186](https://github.com/omarroth/invidious/pull/186). This is an enormous improvement in portability for the project, and opens the door for Heroku support (see [#162](https://github.com/omarroth/invidious/issues/162)), and seamless support on Windows. For most users, it should be as easy as running `docker-compose up`.
+
+I've spent quite a bit of time this past week improving support for geo-bypass (see [#92](https://github.com/omarroth/invidious/issues/92)), and am happy to note that Invidious has been able to proxy ~50% of the geo-restricted videos I've tried. In addition, you can now watch geo-restricted videos if you have `dash` enabled as your `preferred quality`, for more details see [#34](https://github.com/omarroth/invidious/issues/34) and [#185](https://github.com/omarroth/invidious/issues/185), or last week's update. For folks interested in replicating these results for themselves, I'd take a look [here](https://gist.github.com/omarroth/3ce0f276c43e0c4b13e7d9cd35524688) for the script used, and [here](https://gist.github.com/omarroth/beffc4a76a7b82a422e1b36a571878ef) for a list of videos restricted in the US.
+
+1080p has seen a fairly smooth roll-out, although there have been a couple issues reported, mainly [#193](https://github.com/omarroth/invidious/issues/193), which is likely an issue in the player. I've also encountered a couple other issues myself that I would like to investigate. Although none are major, I'd like to keep 1080p opt-in for registered users another week to better address these issues.
+
+Have an excellent week everyone.
+
+# 0.7.0 (2018-09-25)
+
+## Week 7: 1080p and Search Types
+
+Hello again everyone! I've got quite a couple announcements this week:
+
+Experimental 1080p support has been added with [`b3ca392`](https://github.com/omarroth/invidious/b3ca392), and can be enabled by going to preferences and changing `preferred video quality` to `dash`. You can find more details [here](https://github.com/omarroth/invidious/issues/34#issuecomment-424171888). Currently quality and speed controls have not yet been integrated into the player, but I'd still appreciate feedback, mainly on any issues with buffering or DASH playback. I hope to integrate 1080p support into the player and push support site-wide in the coming weeks.
+
+You can now filter content types in search with the `type:TYPE` filter. Supported content types are `playlist`, `channel`, and `video`. More info is available [here](https://github.com/omarroth/invidious/issues/126#issuecomment-423823148). I think this is quite an improvement in usability and I hope others find the same.
+
+A [CHANGELOG](https://github.com/omarroth/invidious/blob/master/CHANGELOG.md) has been added to the repository, so folks will now receive a copy of all these updates when cloning. I think this is an improvement in hosting the project, as it is no longer tied to the `/releases` tab on Github or the posts on Patreon.
+
+Recently, users have been reporting 504s when attempting to access their subscriptions, which is tracked in [#173](https://github.com/omarroth/invidious/issues/173). This is most likely caused by an uptick in usage, which I am absolutely grateful for, but unfortunately has resulted in an increase in costs for hosting the site, which is why I will be bumping my goal on Patreon from $60 to $80. I would appreciate any feedback on how subscriptions could be improved.
+
+Other minor improvements include:
+- Additional regions added to bypass geo-block with [`9a78523`](https://github.com/omarroth/invidious/9a78523)
+- Fix for playlists containing less than 100 videos (previously shown as empty) with [`35ac887`](https://github.com/omarroth/invidious/35ac887)
+- Fix for `published` date for Reddit comments (previously showing negative seconds) with [`6e09202`](https://github.com/omarroth/invidious/6e09202)
+
+Thank you everyone for your support!
+
+# 0.6.0 (2018-09-18)
+
+## Week 6: Filters and Thumbnails
+
 Hello again! This week I'm happy to mention a couple new features to search as well as some miscellaneous usability improvements.
 
 You can now constrain your search query to a specific channel with the `channel:CHANNEL` filter (see [#165](https://github.com/omarroth/invidious/issues/165) for more details). Unfortunately, other search filters combined with channel search are not yet supported. I hope to add support for them in the coming weeks. 
@@ -14,12 +51,12 @@ As a smaller improvement to the site, you can also now view RSS feeds for playli
 
 These updates are also now listed under Github's [releases](https://github.com/omarroth/invidious/releases). I'm also planning on adding them as a `CHANGELOG.md` in the repository itself so people can receive a copy with the project's source.
 
-That's all for this week. Thank you everyone for your support!
-
-# 0.5.0 (2018-09-11)
-
-## Week 5: Privacy and Security
-
+That's all for this week. Thank you everyone for your support!
+
+# 0.5.0 (2018-09-11)
+
+## Week 5: Privacy and Security
+
 I hope everyone had a good weekend! This past week I've been fixing some issues that have been brought to my attention to help better protect users and help them keep their anonymity.
 
 An issue with open referers has been fixed with [`29a2186`](https://github.com/omarroth/invidious/29a2186), which prevents potential redirects to external sites on actions such as login or modifying preferences.
@@ -46,12 +83,12 @@ Folks have also probably noticed that the gutters on either side of the screen h
 "Music", "Sports", and "Popular on YouTube" channels now properly display their videos. You can subscribe to these channels just as you would normally.
 
 This coming week I'm planning on spending time with my family, so I unfortunately may not be as responsive. I do still hope to add some smaller features for next week however, and I hope to continue development soon.
-Thank you everyone again for your support.
-
-# 0.4.0 (2018-09-06)
-
-## Week 4: Genre Channels
-
+Thank you everyone again for your support.
+
+# 0.4.0 (2018-09-06)
+
+## Week 4: Genre Channels
+
 Hello! I hope everyone enjoyed their weekend. Without further ado:
 Just today genre channels have been added with [#119](https://github.com/omarroth/invidious/issues/119). More information on genre channels is available [here](https://support.google.com/youtube/answer/2579942). You can subscribe to them as normally, and view them as RSS. I think they offer an interesting alternative way to find new content and I hope people find them useful.
 
@@ -63,12 +100,12 @@ One of the major use cases for Invidious is as a stripped-down version of YouTub
 
 Finally, I'm pleased to announce that Invidious has hit 100 stars on GitHub. I am very happy that Invidious has proven to be useful to so many people, and I can't say how grateful I am to everyone for their continued support.
 
-Enjoy the rest of your week everyone!
-
-# 0.3.0 (2018-09-06)
-
-## Week 3: Quality of Life
-
+Enjoy the rest of your week everyone!
+
+# 0.3.0 (2018-09-06)
+
+## Week 3: Quality of Life
+
 Hello everyone! This week I've been working on some smaller features that will hopefully make the site more functional.
 Search filters have been added with [#126](https://github.com/omarroth/invidious/issues/126). You can now specify 'sort', 'date', 'duration', and 'features' within your query using the 'operator:value' syntax. I'd recommend taking a look [here](https://github.com/omarroth/invidious/blob/master/src/invidious/search.cr#L33-L114) for a list of supported options and at [#126](https://github.com/omarroth/invidious/issues/126) for some examples. This also opens the door for features such as [#30](https://github.com/omarroth/invidious/issues/30) which can be implemented as filters. I think advanced search is a major point in which Invidious can improve on YouTube and hope to add more features soon!
 
@@ -80,12 +117,12 @@ I'd also like to announce that I've set up an account on [Liberapay](https://lib
 
 [Two weeks ago](https://github.com/omarroth/invidious/releases/tag/0.1.0) I mentioned adding 1080p support into the player. Currently, the only thing blocking is [#207](https://github.com/videojs/http-streaming/pull/207) in the excellent [http-streaming](https://github.com/videojs/http-streaming) library. I hope to work with the videojs team to merge it soon and finally implement 1080p support!
 
-That's all for this week, thank you again everyone for your support! 
-
-# 0.2.0 (2018-09-06)
-
-## Week 2: Toward Playlists
-
+That's all for this week, thank you again everyone for your support! 
+
+# 0.2.0 (2018-09-06)
+
+## Week 2: Toward Playlists
+
 Sorry for the late update! Not as much to announce this week, but still a couple things of note:
 I'm happy to announce that a playlists page and API endpoint has been added so you can now view playlists. Currently, you cannot watch playlists through the player, but I hope to add that in the coming week as well as adding functionality to add and modify playlists. There is a good conversation on [#114](https://github.com/omarroth/invidious/issues/114) about giving playlists even more functionality, which I think is interesting and would appreciate feedback on.
 
@@ -99,12 +136,12 @@ A couple of miscellaneous features and bugfixes:
 
 - Changed YouTube comment header to "View x comments" - [#120](https://github.com/omarroth/invidious/issues/120) 
 
-Enjoy your week everyone!
-
-# 0.1.0 (2018-09-06)
-
-## Week 1: Invidious API and Geo-Bypass
-
+Enjoy your week everyone!
+
+# 0.1.0 (2018-09-06)
+
+## Week 1: Invidious API and Geo-Bypass
+
 Hello everyone! This past week there have been quite a few things worthy of mention:
 
 I'm happy to announce the [Invidious Developer API](https://github.com/omarroth/invidious/wiki/API). The Invidious API does not use any of the official YouTube APIs, and instead crawls the site to provide a JSON interface for other developers to use. It's still under development but is already powering [CloudTube](https://github.com/cloudrac3r/cadencegq). The API currently does not have a quota (compared to YouTube) which I hope to continue thanks to continued support from my Patrons. Hopefully other developers find it useful, and I hope to continue to improve it so it can better serve the community.
@@ -113,4 +150,4 @@ Just today partial support for bypassing geo-restrictions has been added with [f
 
 Support for generating DASH manifests has been fixed, in the coming week I hope to integrate this functionality into the watch page, so users can view videos in 1080p and above.
 
-Thank you everyone for your continued interest and support!
+Thank you everyone for your continued interest and support!

README.md

@@ -28,6 +28,29 @@ BCH: qq4ptclkzej5eza6a50et5ggc58hxsq5aylqut2npk
 
 ## Installation
 
+### Docker:
+
+#### Build and start cluster:
+
+```bash
+$ docker-compose up
+```
+
+And visit `localhost:3000` in your browser.
+
+#### Rebuild cluster:
+
+```bash
+$ docker-compose build
+```
+
+#### Delete data and rebuild:
+
+```bash
+$ docker volume rm invidious_postgresdata
+$ docker-compose build
+```
+
 ### Installing [Crystal](https://github.com/crystal-lang/crystal):
 
 #### On Arch:
@@ -74,8 +97,21 @@ $ sudo pacman -S imagemagick librsvg
 ## Usage:
 
 ```bash
-$ crystal build src/invidious.cr
-$ ./invidious
+$ crystal build src/invidious.cr --release
+$ ./invidious -h
+Usage: invidious [arguments]
+    -b HOST, --bind HOST             Host to bind (defaults to 0.0.0.0)
+    -p PORT, --port PORT             Port to listen for connections (defaults to 3000)
+    -s, --ssl                        Enables SSL
+    --ssl-key-file FILE              SSL key file
+    --ssl-cert-file FILE             SSL certificate file
+    -h, --help                       Shows this help
+    -t THREADS, --crawl-threads=THREADS
+                                     Number of threads for crawling (default: 1)
+    -c THREADS, --channel-threads=THREADS
+                                     Number of threads for refreshing channels (default: 1)
+    -v THREADS, --video-threads=THREADS
+                                     Number of threads for refreshing videos (default: 1)
 ```
 
 Or for development:

assets/css/default.css

@@ -17,6 +17,11 @@ div {
   animation: spin 2s linear infinite;
 }
 
+.playlist-restricted {
+  height: 20em;
+  padding-right: 10px;
+}
+
 /*
  * Navbar
  */

assets/js/watch.js

@@ -0,0 +1,48 @@
+function toggle_parent(target) {
+  body = target.parentNode.parentNode.children[1];
+  if (body.style.display === null || body.style.display === "") {
+    target.innerHTML = "[ + ]";
+    body.style.display = "none";
+  } else {
+    target.innerHTML = "[ - ]";
+    body.style.display = "";
+  }
+}
+
+function toggle_comments(target) {
+  body = target.parentNode.parentNode.parentNode.children[1];
+  if (body.style.display === null || body.style.display === "") {
+    target.innerHTML = "[ + ]";
+    body.style.display = "none";
+  } else {
+    target.innerHTML = "[ - ]";
+    body.style.display = "";
+  }
+}
+
+function swap_comments(source) {
+  comments = document.getElementById("comments");
+  var fallback = comments.innerHTML;
+  comments.innerHTML =
+    '<h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>';
+
+  if (source == "youtube") {
+    get_youtube_comments();
+  } else if (source == "reddit") {
+    get_reddit_comments();
+  }
+}
+
+function commaSeparateNumber(val) {
+  while (/(\d+)(\d{3})/.test(val.toString())) {
+    val = val.toString().replace(/(\d+)(\d{3})/, "$1" + "," + "$2");
+  }
+  return val;
+}
+
+String.prototype.supplant = function(o) {
+  return this.replace(/{([^{}]*)}/g, function(a, b) {
+    var r = o[b];
+    return typeof r === "string" || typeof r === "number" ? r : a;
+  });
+};

config/config.yml

@@ -8,4 +8,5 @@ db:
   port: 5432
   dbname: invidious
 full_refresh: false
-https_only: false
+https_only: false
+geo_bypass: true

docker-compose.yml

@@ -0,0 +1,21 @@
+version: '3'
+services:
+  postgres:
+    build:
+      context: .
+      dockerfile: docker/Dockerfile.postgres
+    restart: unless-stopped
+    volumes:
+      - postgresdata:/var/lib/postgresql/data
+  invidious:
+    build:
+      context: .
+      dockerfile: docker/Dockerfile
+    restart: unless-stopped
+    ports:
+      - "3000:3000"
+    depends_on:
+      - postgres
+
+volumes:
+  postgresdata:

docker/Dockerfile

@@ -0,0 +1,15 @@
+FROM archlinux/base
+
+RUN pacman -Sy --noconfirm shards crystal imagemagick librsvg \
+    which pkgconf gcc ttf-liberation
+# base-devel contains many other basic packages, that are normally assumed to already exist on a clean arch system
+
+ADD . /invidious
+
+WORKDIR /invidious
+
+RUN sed -i 's/host: localhost/host: postgres/' config/config.yml && \
+    shards && \
+    crystal build src/invidious.cr
+
+CMD [ "/invidious/invidious" ]

docker/Dockerfile.postgres

@@ -0,0 +1,10 @@
+FROM postgres:10
+
+ENV POSTGRES_USER postgres
+
+ADD ./setup.sh /setup.sh
+ADD ./config/sql /config/sql
+ADD ./docker/entrypoint.postgres.sh /entrypoint.sh
+
+ENTRYPOINT [ "/entrypoint.sh" ]
+CMD [ "postgres" ]

docker/entrypoint.postgres.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+
+CMD="$@"
+if [ ! -f /var/lib/postgresql/data/setupFinished ]; then
+    echo "### first run - setting up invidious database"
+    /usr/local/bin/docker-entrypoint.sh postgres &
+    sleep 10
+    until runuser -l postgres -c 'pg_isready' 2>/dev/null; do
+        >&2 echo "### Postgres is unavailable - waiting"
+        sleep 5
+    done
+    >&2 echo "### importing table schemas"
+    su postgres -c "/setup.sh" && touch /var/lib/postgresql/data/setupFinished
+    echo "### invidious database setup finished"
+    exit
+fi
+
+echo "running postgres /usr/local/bin/docker-entrypoint.sh $CMD"
+exec /usr/local/bin/docker-entrypoint.sh $CMD

setup.sh

@@ -1,7 +1,8 @@
 #!/bin/bash
 
 createdb invidious
-createuser kemal
+#createuser kemal
+psql -c "CREATE USER kemal WITH PASSWORD 'kemal';"
 psql invidious < config/sql/channels.sql
 psql invidious < config/sql/videos.sql
 psql invidious < config/sql/channel_videos.sql

shard.yml

@@ -1,5 +1,5 @@
 name: invidious
-version: 0.7.0
+version: 0.8.0
 
 authors:
   - Omar Roth <omarroth@hotmail.com>

src/invidious.cr

@@ -105,6 +105,17 @@ spawn do
   end
 end
 
+proxies = {} of String => Array({ip: String, port: Int32})
+if CONFIG.geo_bypass
+  spawn do
+    find_working_proxies(BYPASS_REGIONS) do |region, list|
+      if !list.empty?
+        proxies[region] = list
+      end
+    end
+  end
+end
+
 before_all do |env|
   env.response.headers["X-XSS-Protection"] = "1; mode=block;"
   env.response.headers["X-Content-Type-Options"] = "nosniff"
@@ -206,6 +217,8 @@ get "/watch" do |env|
     next env.redirect "/"
   end
 
+  plid = env.params.query["list"]?
+
   user = env.get? "user"
   if user
     user = user.as(User)
@@ -225,7 +238,9 @@ get "/watch" do |env|
   end
 
   begin
-    video = get_video(id, PG_DB)
+    video = get_video(id, PG_DB, proxies)
+  rescue ex : VideoRedirect
+    next env.redirect "/watch?v=#{ex.message}"
   rescue ex
     error_message = ex.message
     STDOUT << id << " : " << ex.message << "\n"
@@ -325,7 +340,9 @@ get "/embed/:id" do |env|
   params = process_video_params(env.params.query, nil)
 
   begin
-    video = get_video(id, PG_DB)
+    video = get_video(id, PG_DB, proxies)
+  rescue ex : VideoRedirect
+    next env.redirect "/embed/#{ex.message}"
   rescue ex
     error_message = ex.message
     next templated "error"
@@ -381,6 +398,7 @@ get "/embed/:id" do |env|
 end
 
 # Playlists
+
 get "/playlist" do |env|
   plid = env.params.query["list"]?
   if !plid
@@ -390,17 +408,45 @@ get "/playlist" do |env|
   page = env.params.query["page"]?.try &.to_i?
   page ||= 1
 
+  if plid.starts_with? "RD"
+    next env.redirect "/mix?list=#{plid}"
+  end
+
   begin
     playlist = fetch_playlist(plid)
-    videos = fetch_playlist_videos(plid, page, playlist.video_count)
   rescue ex
     error_message = ex.message
     next templated "error"
   end
 
+  begin
+    videos = fetch_playlist_videos(plid, page, playlist.video_count)
+  rescue ex
+    videos = [] of PlaylistVideo
+  end
+
   templated "playlist"
 end
 
+get "/mix" do |env|
+  rdid = env.params.query["list"]?
+  if !rdid
+    next env.redirect "/"
+  end
+
+  continuation = env.params.query["continuation"]?
+  continuation ||= rdid.lchop("RD")
+
+  begin
+    mix = fetch_mix(rdid, continuation)
+  rescue ex
+    error_message = ex.message
+    next templated "error"
+  end
+
+  templated "mix"
+end
+
 # Search
 
 get "/results" do |env|
@@ -1079,12 +1125,14 @@ post "/data_control" do |env|
         body = JSON.parse(body)
         body["subscriptions"].as_a.each do |ucid|
           ucid = ucid.as_s
-          if !user.subscriptions.includes? ucid
-            PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE id = $2", ucid, user.id)
 
+          if !user.subscriptions.includes? ucid
             begin
               client = make_client(YT_URL)
               get_channel(ucid, client, PG_DB, false, false)
+
+              PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
+              user.subscriptions << ucid
             rescue ex
               next
             end
@@ -1093,8 +1141,10 @@ post "/data_control" do |env|
 
         body["watch_history"].as_a.each do |id|
           id = id.as_s
+
           if !user.watched.includes? id
             PG_DB.exec("UPDATE users SET watched = array_append(watched,$1) WHERE email = $2", id, user.email)
+            user.watched << id
           end
         end
 
@@ -1105,11 +1155,12 @@ post "/data_control" do |env|
           ucid = channel["xmlUrl"].match(/UC[a-zA-Z0-9_-]{22}/).not_nil![0]
 
           if !user.subscriptions.includes? ucid
-            PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
-
             begin
               client = make_client(YT_URL)
               get_channel(ucid, client, PG_DB, false, false)
+
+              PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
+              user.subscriptions << ucid
             rescue ex
               next
             end
@@ -1120,11 +1171,12 @@ post "/data_control" do |env|
           ucid = md["channel_id"]
 
           if !user.subscriptions.includes? ucid
-            PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
-
             begin
               client = make_client(YT_URL)
               get_channel(ucid, client, PG_DB, false, false)
+
+              PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
+              user.subscriptions << ucid
             rescue ex
               next
             end
@@ -1136,11 +1188,12 @@ post "/data_control" do |env|
           ucid = channel["url"].as_s.match(/UC[a-zA-Z0-9_-]{22}/).not_nil![0]
 
           if !user.subscriptions.includes? ucid
-            PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
-
             begin
               client = make_client(YT_URL)
               get_channel(ucid, client, PG_DB, false, false)
+
+              PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
+              user.subscriptions << ucid
             rescue ex
               next
             end
@@ -1156,19 +1209,24 @@ post "/data_control" do |env|
 
               db = entry.io.gets_to_end
               db.scan(/youtube\.com\/watch\?v\=(?<id>[a-zA-Z0-9_-]{11})/) do |md|
-                if !user.watched.includes? md["id"]
-                  PG_DB.exec("UPDATE users SET watched = array_append(watched,$1) WHERE email = $2", md["id"], user.email)
+                id = md["id"]
+
+                if !user.watched.includes? id
+                  PG_DB.exec("UPDATE users SET watched = array_append(watched,$1) WHERE email = $2", id, user.email)
+                  user.watched << id
                 end
               end
 
               db.scan(/youtube\.com\/channel\/(?<ucid>[a-zA-Z0-9_-]{22})/) do |md|
                 ucid = md["ucid"]
-                if !user.subscriptions.includes? ucid
-                  PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
 
+                if !user.subscriptions.includes? ucid
                   begin
                     client = make_client(YT_URL)
                     get_channel(ucid, client, PG_DB, false, false)
+
+                    PG_DB.exec("UPDATE users SET subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
+                    user.subscriptions << ucid
                   rescue ex
                     next
                   end
@@ -1718,11 +1776,15 @@ end
 # API Endpoints
 
 get "/api/v1/captions/:id" do |env|
+  env.response.content_type = "application/json"
+
   id = env.params.url["id"]
 
   client = make_client(YT_URL)
   begin
-    video = get_video(id, PG_DB)
+    video = get_video(id, PG_DB, proxies)
+  rescue ex : VideoRedirect
+    next env.redirect "/api/v1/captions/#{ex.message}"
   rescue ex
     halt env, status_code: 403
   end
@@ -1730,9 +1792,10 @@ get "/api/v1/captions/:id" do |env|
   captions = video.captions
 
   label = env.params.query["label"]?
-  if !label
-    env.response.content_type = "application/json"
+  lang = env.params.query["lang"]?
+  tlang = env.params.query["tlang"]?
 
+  if !label && !lang
     response = JSON.build do |json|
       json.object do
         json.field "captions" do
@@ -1752,22 +1815,27 @@ get "/api/v1/captions/:id" do |env|
     next response
   end
 
+  env.response.content_type = "text/vtt"
+
   caption = captions.select { |caption| caption.name.simpleText == label }
 
-  env.response.content_type = "text/vtt"
+  if lang
+    caption = captions.select { |caption| caption.languageCode == lang }
+  end
+
   if caption.empty?
-    halt env, status_code: 403
+    halt env, status_code: 404
   else
     caption = caption[0]
   end
 
-  caption_xml = client.get(caption.baseUrl).body
+  caption_xml = client.get(caption.baseUrl + "&tlang=#{tlang}").body
   caption_xml = XML.parse(caption_xml)
 
   webvtt = <<-END_VTT
   WEBVTT
   Kind: captions
-  Language: #{caption.languageCode}
+  Language: #{tlang || caption.languageCode}
 
 
   END_VTT
@@ -1806,6 +1874,8 @@ get "/api/v1/captions/:id" do |env|
 end
 
 get "/api/v1/comments/:id" do |env|
+  env.response.content_type = "application/json"
+
   id = env.params.url["id"]
 
   source = env.params.query["source"]?
@@ -1816,26 +1886,70 @@ get "/api/v1/comments/:id" do |env|
 
   if source == "youtube"
     client = make_client(YT_URL)
+    html = client.get("/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en&disable_polymer=1")
     headers = HTTP::Headers.new
-    html = client.get("/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&disable_polymer=1")
-
     headers["cookie"] = html.cookies.add_request_headers(headers)["cookie"]
-    headers["content-type"] = "application/x-www-form-urlencoded"
-
-    headers["x-client-data"] = "CIi2yQEIpbbJAQipncoBCNedygEIqKPKAQ=="
-    headers["x-spf-previous"] = "https://www.youtube.com/watch?v=#{id}"
-    headers["x-spf-referer"] = "https://www.youtube.com/watch?v=#{id}"
-
-    headers["x-youtube-client-name"] = "1"
-    headers["x-youtube-client-version"] = "2.20180719"
-
     body = html.body
+
     session_token = body.match(/'XSRF_TOKEN': "(?<session_token>[A-Za-z0-9\_\-\=]+)"/).not_nil!["session_token"]
+    itct = body.match(/itct=(?<itct>[^"]+)"/).not_nil!["itct"]
     ctoken = body.match(/'COMMENTS_TOKEN': "(?<ctoken>[^"]+)"/)
 
-    if !ctoken
-      env.response.content_type = "application/json"
+    if body.match(/<meta itemprop="regionsAllowed" content="">/)
+      bypass_channel = Channel({String, HTTPClient, HTTP::Headers} | Nil).new
 
+      proxies.each do |region, list|
+        spawn do
+          proxy_html = %(<meta itemprop="regionsAllowed" content="">)
+
+          list.each do |proxy|
+            begin
+              proxy_client = HTTPClient.new(YT_URL)
+              proxy_client.read_timeout = 10.seconds
+              proxy_client.connect_timeout = 10.seconds
+
+              proxy = list.sample(1)[0]
+              proxy = HTTPProxy.new(proxy_host: proxy[:ip], proxy_port: proxy[:port])
+              proxy_client.set_proxy(proxy)
+
+              response = proxy_client.get("/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en&disable_polymer=1")
+              proxy_headers = HTTP::Headers.new
+              proxy_headers["cookie"] = response.cookies.add_request_headers(headers)["cookie"]
+              proxy_html = response.body
+
+              if proxy_html.match(/<meta itemprop="regionsAllowed" content="">/)
+                bypass_channel.send(nil)
+              else
+                bypass_channel.send({proxy_html, proxy_client, proxy_headers})
+              end
+
+              break
+            rescue ex
+            end
+          end
+
+          # If none of the proxies we tried returned a valid response
+          if proxy_html.match(/<meta itemprop="regionsAllowed" content="">/)
+            bypass_channel.send(nil)
+          end
+        end
+      end
+
+      proxies.size.times do
+        response = bypass_channel.receive
+        if response
+          session_token = response[0].match(/'XSRF_TOKEN': "(?<session_token>[A-Za-z0-9\_\-\=]+)"/).not_nil!["session_token"]
+          itct = response[0].match(/itct=(?<itct>[^"]+)"/).not_nil!["itct"]
+          ctoken = response[0].match(/'COMMENTS_TOKEN': "(?<ctoken>[^"]+)"/)
+
+          client = response[1]
+          headers = response[2]
+          break
+        end
+      end
+    end
+
+    if !ctoken
       if format == "json"
         next {"comments" => [] of String}.to_json
       else
@@ -1843,7 +1957,6 @@ get "/api/v1/comments/:id" do |env|
       end
     end
     ctoken = ctoken["ctoken"]
-    itct = body.match(/itct=(?<itct>[^"]+)"/).not_nil!["itct"]
 
     if env.params.query["continuation"]? && !env.params.query["continuation"].empty?
       continuation = env.params.query["continuation"]
@@ -1857,10 +1970,16 @@ get "/api/v1/comments/:id" do |env|
     }
     post_req = HTTP::Params.encode(post_req)
 
-    response = client.post("/comment_service_ajax?action_get_comments=1&pbj=1&ctoken=#{ctoken}&continuation=#{continuation}&itct=#{itct}", headers, post_req).body
-    response = JSON.parse(response)
+    headers["content-type"] = "application/x-www-form-urlencoded"
 
-    env.response.content_type = "application/json"
+    headers["x-client-data"] = "CIi2yQEIpbbJAQipncoBCNedygEIqKPKAQ=="
+    headers["x-spf-previous"] = "https://www.youtube.com/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en&disable_polymer=1"
+    headers["x-spf-referer"] = "https://www.youtube.com/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en&disable_polymer=1"
+
+    headers["x-youtube-client-name"] = "1"
+    headers["x-youtube-client-version"] = "2.20180719"
+    response = client.post("/comment_service_ajax?action_get_comments=1&pbj=1&ctoken=#{ctoken}&continuation=#{continuation}&itct=#{itct}&hl=en&gl=US", headers, post_req)
+    response = JSON.parse(response.body)
 
     if !response["response"]["continuationContents"]?
       halt env, status_code: 403
@@ -2016,8 +2135,6 @@ get "/api/v1/comments/:id" do |env|
       halt env, status_code: 404
     end
 
-    env.response.content_type = "application/json"
-
     if format == "json"
       reddit_thread = JSON.parse(reddit_thread.to_json).as_h
       reddit_thread["comments"] = JSON.parse(comments.to_json)
@@ -2038,7 +2155,7 @@ get "/api/v1/insights/:id" do |env|
 
   client = make_client(YT_URL)
   headers = HTTP::Headers.new
-  html = client.get("/watch?v=#{id}&disable_polymer=1")
+  html = client.get("/watch?v=#{id}&gl=US&hl=en&disable_polymer=1")
 
   headers["cookie"] = html.cookies.add_request_headers(headers)["cookie"]
   headers["content-type"] = "application/x-www-form-urlencoded"
@@ -2113,12 +2230,15 @@ get "/api/v1/insights/:id" do |env|
 end
 
 get "/api/v1/videos/:id" do |env|
+  env.response.content_type = "application/json"
+
   id = env.params.url["id"]
 
   begin
-    video = get_video(id, PG_DB)
+    video = get_video(id, PG_DB, proxies)
+  rescue ex : VideoRedirect
+    next env.redirect "/api/v1/videos/#{ex.message}"
   rescue ex
-    env.response.content_type = "application/json"
     error_message = {"error" => ex.message}.to_json
     halt env, status_code: 500, response: error_message
   end
@@ -2128,7 +2248,6 @@ get "/api/v1/videos/:id" do |env|
 
   captions = video.captions
 
-  env.response.content_type = "application/json"
   video_info = JSON.build do |json|
     json.object do
       json.field "title", video.title
@@ -2822,14 +2941,28 @@ get "/api/v1/playlists/:plid" do |env|
   page = env.params.query["page"]?.try &.to_i?
   page ||= 1
 
+  format = env.params.query["format"]?
+  format ||= "json"
+
+  continuation = env.params.query["continuation"]?
+
+  if plid.starts_with? "RD"
+    next env.redirect "/api/v1/mixes/#{plid}"
+  end
+
   begin
     playlist = fetch_playlist(plid)
-    videos = fetch_playlist_videos(plid, page, playlist.video_count)
   rescue ex
     error_message = {"error" => "Playlist is empty"}.to_json
     halt env, status_code: 404, response: error_message
   end
 
+  begin
+    videos = fetch_playlist_videos(plid, page, playlist.video_count, continuation)
+  rescue ex
+    videos = [] of PlaylistVideo
+  end
+
   response = JSON.build do |json|
     json.object do
       json.field "title", playlist.title
@@ -2839,6 +2972,20 @@ get "/api/v1/playlists/:plid" do |env|
       json.field "authorId", playlist.ucid
       json.field "authorUrl", "/channel/#{playlist.ucid}"
 
+      json.field "authorThumbnails" do
+        json.array do
+          qualities = [32, 48, 76, 100, 176, 512]
+
+          qualities.each do |quality|
+            json.object do
+              json.field "url", playlist.author_thumbnail.gsub("=s100-", "=s#{quality}-")
+              json.field "width", quality
+              json.field "height", quality
+            end
+          end
+        end
+      end
+
       json.field "description", playlist.description
       json.field "descriptionHtml", playlist.description_html
       json.field "videoCount", playlist.video_count
@@ -2870,6 +3017,80 @@ get "/api/v1/playlists/:plid" do |env|
     end
   end
 
+  if format == "html"
+    response = JSON.parse(response)
+    playlist_html = template_playlist(response)
+    next_video = response["videos"].as_a[1]?.try &.["videoId"]
+
+    response = {
+      "playlistHtml" => playlist_html,
+      "nextVideo"    => next_video,
+    }.to_json
+  end
+
+  response
+end
+
+get "/api/v1/mixes/:rdid" do |env|
+  env.response.content_type = "application/json"
+
+  rdid = env.params.url["rdid"]
+
+  continuation = env.params.query["continuation"]?
+  continuation ||= rdid.lchop("RD")
+
+  format = env.params.query["format"]?
+  format ||= "json"
+
+  begin
+    mix = fetch_mix(rdid, continuation)
+  rescue ex
+    error_message = {"error" => ex.message}.to_json
+    halt env, status_code: 500, response: error_message
+  end
+
+  response = JSON.build do |json|
+    json.object do
+      json.field "title", mix.title
+      json.field "mixId", mix.id
+
+      json.field "videos" do
+        json.array do
+          mix.videos.each do |video|
+            json.object do
+              json.field "title", video.title
+              json.field "videoId", video.id
+              json.field "author", video.author
+
+              json.field "authorId", video.ucid
+              json.field "authorUrl", "/channel/#{video.ucid}"
+
+              json.field "videoThumbnails" do
+                json.array do
+                  generate_thumbnails(json, video.id)
+                end
+              end
+
+              json.field "index", video.index
+              json.field "lengthSeconds", video.length_seconds
+            end
+          end
+        end
+      end
+    end
+  end
+
+  if format == "html"
+    response = JSON.parse(response)
+    playlist_html = template_mix(response)
+    next_video = response["videos"].as_a[1]?.try &.["videoId"]
+
+    response = {
+      "playlistHtml" => playlist_html,
+      "nextVideo"    => next_video,
+    }.to_json
+  end
+
   response
 end
 
@@ -2892,7 +3113,9 @@ get "/api/manifest/dash/id/:id" do |env|
 
   client = make_client(YT_URL)
   begin
-    video = get_video(id, PG_DB)
+    video = get_video(id, PG_DB, proxies)
+  rescue ex : VideoRedirect
+    next env.redirect "/api/manifest/dash/id/#{ex.message}"
   rescue ex
     halt env, status_code: 403
   end
@@ -3078,8 +3301,40 @@ get "/videoplayback" do |env|
   host = "https://r#{fvip}---#{mn}.googlevideo.com"
   url = "/videoplayback?#{query_params.to_s}"
 
-  client = make_client(URI.parse(host))
-  response = client.head(url)
+  if query_params["region"]?
+    client = make_client(URI.parse(host))
+    response = HTTP::Client::Response.new(status_code: 403)
+
+    if !proxies[query_params["region"]]?
+      halt env, status_code: 403
+    end
+
+    proxies[query_params["region"]].each do |proxy|
+      begin
+        client = HTTPClient.new(URI.parse(host))
+        client.read_timeout = 10.seconds
+        client.connect_timeout = 10.seconds
+
+        proxy = HTTPProxy.new(proxy_host: proxy[:ip], proxy_port: proxy[:port])
+        client.set_proxy(proxy)
+
+        response = client.head(url)
+        if response.status_code == 200
+          # For whatever reason the proxy needs to be set again
+          client.set_proxy(proxy)
+          break
+        end
+      rescue ex
+      end
+    end
+  else
+    client = make_client(URI.parse(host))
+    response = client.head(url)
+  end
+
+  if response.status_code != 200
+    halt env, status_code: 403
+  end
 
   if response.headers["Location"]?
     url = URI.parse(response.headers["Location"])
@@ -3224,6 +3479,24 @@ get "/vi/:id/:name" do |env|
 end
 
 error 404 do |env|
+  if md = env.request.path.match(/^\/(?<id>[a-zA-Z0-9_-]{11})/)
+    id = md["id"]
+
+    params = [] of String
+    env.params.query.each do |k, v|
+      params << "#{k}=#{v}"
+    end
+    params = params.join("&")
+
+    url = "/watch?v=#{id}"
+    if !params.empty?
+      url += "&#{params}"
+    end
+
+    env.response.headers["Location"] = url
+    halt env, status_code: 302
+  end
+
   error_message = "404 Page not found"
   templated "error"
 end

src/invidious/channels.cr

@@ -104,13 +104,17 @@ def fetch_channel(ucid, client, db, pull_all_videos = true)
 
       videos.each do |video|
         ids << video.id
-        db.exec("UPDATE users SET notifications = notifications || $1 \
+
+        # FIXME: Red videos don't provide published date, so the best we can do is ignore them
+        if Time.now - video.published > 1.minute
+          db.exec("UPDATE users SET notifications = notifications || $1 \
           WHERE updated < $2 AND $3 = ANY(subscriptions) AND $1 <> ALL(notifications)", video.id, video.published, video.ucid)
 
-        video_array = video.to_a
-        args = arg_array(video_array)
-        db.exec("INSERT INTO channel_videos VALUES (#{args}) ON CONFLICT (id) DO UPDATE SET title = $2, \
+          video_array = video.to_a
+          args = arg_array(video_array)
+          db.exec("INSERT INTO channel_videos VALUES (#{args}) ON CONFLICT (id) DO UPDATE SET title = $2, \
           published = $3, updated = $4, ucid = $5, author = $6", video_array)
+        end
       end
 
       if count < 30

src/invidious/comments.cr

@@ -104,21 +104,23 @@ def template_youtube_comments(comments)
 
     html += <<-END_HTML
     <div class="pure-g">
-      <div class="pure-u-2-24">
+      <div class="pure-u-4-24 pure-u-md-2-24">
         <img style="width:90%; padding-right:1em; padding-top:1em;" src="#{author_thumbnail}">
       </div>
-      <div class="pure-u-22-24">
+      <div class="pure-u-20-24 pure-u-md-22-24">
         <p>
-            <a href="javascript:void(0)" onclick="toggle(this)">[ - ]</a> 
-            <i class="icon ion-ios-thumbs-up"></i> #{child["likeCount"]} 
-            <b><a href="#{child["authorUrl"]}">#{child["author"]}</a></b> 
-            - #{recode_date(Time.epoch(child["published"].as_i64))} ago
-          </p>
+          <a href="javascript:void(0)" onclick="toggle_parent(this)">[ - ]</a> 
+          <b>
+            <a href="#{child["authorUrl"]}">#{child["author"]}</a>
+          </b> 
           <div>
           <p style="white-space:pre-wrap">#{child["contentHtml"]}</p>
-          #{replies_html}
-          </div>
-        </div>
+          #{recode_date(Time.epoch(child["published"].as_i64))} ago
+          | 
+          <i class="icon ion-ios-thumbs-up"></i> #{child["likeCount"]} 
+        </p>
+        #{replies_html}
+      </div>
     </div>
     END_HTML
   end
@@ -156,10 +158,10 @@ def template_reddit_comments(root)
 
       content = <<-END_HTML
       <p>
-        <a href="javascript:void(0)" onclick="toggle(this)">[ - ]</a> 
-        <i class="icon ion-ios-thumbs-up"></i> #{score} 
+        <a href="javascript:void(0)" onclick="toggle_parent(this)">[ - ]</a> 
         <b><a href="https://www.reddit.com/user/#{author}">#{author}</a></b> 
-        - #{recode_date(child.created_utc)} ago
+        #{score} points 
+        #{recode_date(child.created_utc)} ago
       </p>
       <div>
       #{body_html}

src/invidious/helpers/helpers.cr

@@ -14,6 +14,7 @@ class Config
     https_only:   Bool?,
     hmac_key:     String?,
     full_refresh: Bool,
+    geo_bypass:   Bool,
   })
 end
 
@@ -244,11 +245,22 @@ def extract_items(nodeset, ucid = nil)
       plid = HTTP::Params.parse(URI.parse(id).query.not_nil!)["list"]
 
       anchor = node.xpath_node(%q(.//div[contains(@class, "yt-lockup-meta")]/a))
+
       if !anchor
         anchor = node.xpath_node(%q(.//ul[@class="yt-lockup-meta-info"]/li/a))
       end
-      if anchor
-        video_count = anchor.content.match(/View full playlist \((?<count>\d+)/).try &.["count"].to_i?
+
+      video_count = node.xpath_node(%q(.//span[@class="formatted-video-count-label"]/b))
+      if video_count
+        video_count = video_count.content
+
+        if video_count == "50+"
+          author = "YouTube"
+          author_id = "UC-9-kyTW8ZkZNDHQJ6FgpwQ"
+          video_count = video_count.rchop("+")
+        end
+
+        video_count = video_count.to_i?
       end
       video_count ||= 0
 

src/invidious/helpers/proxy.cr

@@ -89,6 +89,68 @@ class HTTPClient < HTTP::Client
 end
 
 def get_proxies(country_code = "US")
+  # return get_spys_proxies(country_code)
+  return get_nova_proxies(country_code)
+end
+
+def filter_proxies(proxies)
+  proxies.select! do |proxy|
+    begin
+      client = HTTPClient.new(YT_URL)
+      client.read_timeout = 10.seconds
+      client.connect_timeout = 10.seconds
+
+      proxy = HTTPProxy.new(proxy_host: proxy[:ip], proxy_port: proxy[:port])
+      client.set_proxy(proxy)
+
+      client.head("/").status_code == 200
+    rescue ex
+      false
+    end
+  end
+
+  return proxies
+end
+
+def get_nova_proxies(country_code = "US")
+  country_code = country_code.downcase
+  client = HTTP::Client.new(URI.parse("https://www.proxynova.com"))
+  client.read_timeout = 10.seconds
+  client.connect_timeout = 10.seconds
+
+  headers = HTTP::Headers.new
+  headers["User-Agent"] = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"
+  headers["Accept"] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
+  headers["Accept-Language"] = "Accept-Language: en-US,en;q=0.9"
+  headers["Host"] = "www.proxynova.com"
+  headers["Origin"] = "https://www.proxynova.com"
+  headers["Referer"] = "https://www.proxynova.com/proxy-server-list/country-#{country_code}/"
+
+  response = client.get("/proxy-server-list/country-#{country_code}/", headers)
+  document = XML.parse_html(response.body)
+
+  proxies = [] of {ip: String, port: Int32, score: Float64}
+  document.xpath_nodes(%q(//tr[@data-proxy-id])).each do |node|
+    ip = node.xpath_node(%q(.//td/abbr/script)).not_nil!.content
+    ip = ip.match(/document\.write\('(?<sub1>[^']+)'.substr\(8\) \+ '(?<sub2>[^']+)'/).not_nil!
+    ip = "#{ip["sub1"][8..-1]}#{ip["sub2"]}"
+    port = node.xpath_node(%q(.//td[2])).not_nil!.content.strip.to_i
+
+    anchor = node.xpath_node(%q(.//td[4]/div)).not_nil!
+    speed = anchor["data-value"].to_f
+    latency = anchor["title"].to_f
+    uptime = node.xpath_node(%q(.//td[5]/span)).not_nil!.content.rchop("%").to_f
+
+    # TODO: Tweak me
+    score = (uptime*4 + speed*2 + latency)/7
+    proxies << {ip: ip, port: port, score: score}
+  end
+
+  # proxies = proxies.sort_by { |proxy| proxy[:score] }.reverse
+  return proxies
+end
+
+def get_spys_proxies(country_code = "US")
   client = HTTP::Client.new(URI.parse("http://spys.one"))
   client.read_timeout = 10.seconds
   client.connect_timeout = 10.seconds
@@ -108,7 +170,15 @@ def get_proxies(country_code = "US")
     "xf4" => "0",
     "xf5" => "1",
   }
+
   response = client.post("/free-proxy-list/#{country_code}/", headers, form: body)
+  20.times do
+    if response.status_code == 200
+      break
+    end
+    response = client.post("/free-proxy-list/#{country_code}/", headers, form: body)
+  end
+
   response = XML.parse_html(response.body)
 
   mapping = response.xpath_node(%q(.//body/script)).not_nil!.content

src/invidious/jobs.cr

@@ -154,3 +154,16 @@ def update_decrypt_function
     Fiber.yield
   end
 end
+
+def find_working_proxies(regions)
+  loop do
+    regions.each do |region|
+      proxies = get_proxies(region).first(20)
+      proxies = proxies.map { |proxy| {ip: proxy[:ip], port: proxy[:port]} }
+      # proxies = filter_proxies(proxies)
+
+      yield region, proxies
+      Fiber.yield
+    end
+  end
+end

src/invidious/mixes.cr

@@ -0,0 +1,114 @@
+class MixVideo
+  add_mapping({
+    title:          String,
+    id:             String,
+    author:         String,
+    ucid:           String,
+    length_seconds: Int32,
+    index:          Int32,
+    mixes:          Array(String),
+  })
+end
+
+class Mix
+  add_mapping({
+    title:  String,
+    id:     String,
+    videos: Array(MixVideo),
+  })
+end
+
+def fetch_mix(rdid, video_id, cookies = nil)
+  client = make_client(YT_URL)
+  headers = HTTP::Headers.new
+  headers["User-Agent"] = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
+
+  if cookies
+    headers = cookies.add_request_headers(headers)
+  end
+  response = client.get("/watch?v=#{video_id}&list=#{rdid}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en", headers)
+
+  yt_data = response.body.match(/window\["ytInitialData"\] = (?<data>.*);/)
+  if yt_data
+    yt_data = JSON.parse(yt_data["data"].rchop(";"))
+  else
+    raise "Could not create mix."
+  end
+
+  if !yt_data["contents"]["twoColumnWatchNextResults"]["playlist"]?
+    raise "Could not create mix."
+  end
+
+  playlist = yt_data["contents"]["twoColumnWatchNextResults"]["playlist"]["playlist"]
+  mix_title = playlist["title"].as_s
+
+  contents = playlist["contents"].as_a
+  until contents[0]["playlistPanelVideoRenderer"]["videoId"].as_s == video_id
+    contents.shift
+  end
+
+  videos = [] of MixVideo
+  contents.each do |item|
+    item = item["playlistPanelVideoRenderer"]
+
+    id = item["videoId"].as_s
+    title = item["title"]["simpleText"].as_s
+    author = item["longBylineText"]["runs"][0]["text"].as_s
+    ucid = item["longBylineText"]["runs"][0]["navigationEndpoint"]["browseEndpoint"]["browseId"].as_s
+    length_seconds = decode_length_seconds(item["lengthText"]["simpleText"].as_s)
+    index = item["navigationEndpoint"]["watchEndpoint"]["index"].as_i
+
+    videos << MixVideo.new(
+      title,
+      id,
+      author,
+      ucid,
+      length_seconds,
+      index,
+      [rdid]
+    )
+  end
+
+  if !cookies
+    next_page = fetch_mix(rdid, videos[-1].id, response.cookies)
+    videos += next_page.videos
+  end
+
+  videos.uniq! { |video| video.id }
+  videos = videos.first(50)
+  return Mix.new(mix_title, rdid, videos)
+end
+
+def template_mix(mix)
+  html = <<-END_HTML
+  <h3>
+    <a href="/mix?list=#{mix["mixId"]}">
+      #{mix["title"]}
+    </a>
+  </h3>
+  <div class="pure-menu pure-menu-scrollable playlist-restricted">
+    <ol class="pure-menu-list">
+  END_HTML
+
+  mix["videos"].as_a.each do |video|
+    html += <<-END_HTML
+      <li class="pure-menu-item">
+        <a href="/watch?v=#{video["videoId"]}&list=#{mix["mixId"]}">
+          <img style="width:100%;" src="/vi/#{video["videoId"]}/mqdefault.jpg">
+          <p style="width:100%">#{video["title"]}</p>
+          <p>
+              <b style="width: 100%">#{video["author"]}</b>
+          </p>
+        </a>
+      </li>
+    END_HTML
+  end
+
+  html += <<-END_HTML
+    </ol>
+  </div>
+  <hr>
+  END_HTML
+
+  html
+end

src/invidious/playlists.cr

@@ -1,17 +1,3 @@
-class Playlist
-  add_mapping({
-    title:            String,
-    id:               String,
-    author:           String,
-    ucid:             String,
-    description:      String,
-    description_html: String,
-    video_count:      Int32,
-    views:            Int64,
-    updated:          Time,
-  })
-end
-
 class PlaylistVideo
   add_mapping({
     title:          String,
@@ -25,11 +11,38 @@ class PlaylistVideo
   })
 end
 
-def fetch_playlist_videos(plid, page, video_count)
+class Playlist
+  add_mapping({
+    title:            String,
+    id:               String,
+    author:           String,
+    author_thumbnail: String,
+    ucid:             String,
+    description:      String,
+    description_html: String,
+    video_count:      Int32,
+    views:            Int64,
+    updated:          Time,
+  })
+end
+
+def fetch_playlist_videos(plid, page, video_count, continuation = nil)
   client = make_client(YT_URL)
 
-  if video_count > 100
+  if continuation
+    html = client.get("/watch?v=#{continuation}&list=#{plid}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en&disable_polymer=1")
+    html = XML.parse_html(html.body)
+
+    index = html.xpath_node(%q(//span[@id="playlist-current-index"])).try &.content.to_i?
+    if index
+      index -= 1
+    end
+    index ||= 0
+  else
     index = (page - 1) * 100
+  end
+
+  if video_count > 100
     url = produce_playlist_url(plid, index)
 
     response = client.get(url)
@@ -42,10 +55,12 @@ def fetch_playlist_videos(plid, page, video_count)
     nodeset = document.xpath_nodes(%q(.//tr[contains(@class, "pl-video")]))
     videos = extract_playlist(plid, nodeset, index)
   else
+    # Playlist has less than one page of videos, so subsequent pages will be empty
     if page > 1
       videos = [] of PlaylistVideo
     else
-      response = client.get("/playlist?list=#{plid}&disable_polymer=1")
+      # Extract first page of videos
+      response = client.get("/playlist?list=#{plid}&gl=US&hl=en&disable_polymer=1")
       document = XML.parse_html(response.body)
       nodeset = document.xpath_nodes(%q(.//tr[contains(@class, "pl-video")]))
 
@@ -105,7 +120,8 @@ def produce_playlist_url(id, index)
   end
   ucid = "VL" + id
 
-  meta = "\x08#{write_var_int(index).join}"
+  meta = [0x08_u8] + write_var_int(index)
+  meta = Slice.new(meta.to_unsafe, meta.size)
   meta = Base64.urlsafe_encode(meta, false)
   meta = "PT:#{meta}"
 
@@ -141,7 +157,7 @@ def fetch_playlist(plid)
     plid = "UU#{plid.lchop("UC")}"
   end
 
-  response = client.get("/playlist?list=#{plid}&disable_polymer=1")
+  response = client.get("/playlist?list=#{plid}&hl=en&disable_polymer=1")
   if response.status_code != 200
     raise "Invalid playlist."
   end
@@ -160,10 +176,13 @@ def fetch_playlist(plid)
   title = title.content.strip(" \n")
 
   description_html = document.xpath_node(%q(//span[@class="pl-header-description-text"]/div/div[1]))
+  description_html ||= document.xpath_node(%q(//span[@class="pl-header-description-text"]))
   description_html, description = html_to_content(description_html)
 
   anchor = document.xpath_node(%q(//ul[@class="pl-header-details"])).not_nil!
   author = anchor.xpath_node(%q(.//li[1]/a)).not_nil!.content
+  author_thumbnail = document.xpath_node(%q(//img[@class="channel-header-profile-image"])).try &.["src"]
+  author_thumbnail ||= ""
   ucid = anchor.xpath_node(%q(.//li[1]/a)).not_nil!["href"].split("/")[2]
 
   video_count = anchor.xpath_node(%q(.//li[2])).not_nil!.content.delete("videos, ").to_i
@@ -181,6 +200,7 @@ def fetch_playlist(plid)
     title,
     plid,
     author,
+    author_thumbnail,
     ucid,
     description,
     description_html,
@@ -191,3 +211,37 @@ def fetch_playlist(plid)
 
   return playlist
 end
+
+def template_playlist(playlist)
+  html = <<-END_HTML
+  <h3>
+    <a href="/playlist?list=#{playlist["playlistId"]}">
+      #{playlist["title"]}
+    </a>
+  </h3>
+  <div class="pure-menu pure-menu-scrollable playlist-restricted">
+    <ol class="pure-menu-list">
+  END_HTML
+
+  playlist["videos"].as_a.each do |video|
+    html += <<-END_HTML
+      <li class="pure-menu-item">
+        <a href="/watch?v=#{video["videoId"]}&list=#{playlist["playlistId"]}">
+          <img style="width:100%;" src="/vi/#{video["videoId"]}/mqdefault.jpg">
+          <p style="width:100%">#{video["title"]}</p>
+          <p>
+              <b style="width: 100%">#{video["author"]}</b>
+          </p>
+        </a>
+      </li>
+    END_HTML
+  end
+
+  html += <<-END_HTML
+    </ol>
+  </div>
+  <hr>
+  END_HTML
+
+  html
+end

src/invidious/search.cr

@@ -89,7 +89,7 @@ def search(query, page = 1, search_params = produce_search_params(content_type:
     return {0, [] of SearchItem}
   end
 
-  html = client.get("/results?q=#{URI.escape(query)}&page=#{page}&sp=#{search_params}&disable_polymer=1").body
+  html = client.get("/results?q=#{URI.escape(query)}&page=#{page}&sp=#{search_params}&hl=en&disable_polymer=1").body
   if html.empty?
     return {0, [] of SearchItem}
   end

src/invidious/videos.cr

@@ -110,7 +110,7 @@ CAPTION_LANGUAGES = {
 
 REGIONS        = {"AD", "AE", "AF", "AG", "AI", "AL", "AM", "AO", "AQ", "AR", "AS", "AT", "AU", "AW", "AX", "AZ", "BA", "BB", "BD", "BE", "BF", "BG", "BH", "BI", "BJ", "BL", "BM", "BN", "BO", "BQ", "BR", "BS", "BT", "BV", "BW", "BY", "BZ", "CA", "CC", "CD", "CF", "CG", "CH", "CI", "CK", "CL", "CM", "CN", "CO", "CR", "CU", "CV", "CW", "CX", "CY", "CZ", "DE", "DJ", "DK", "DM", "DO", "DZ", "EC", "EE", "EG", "EH", "ER", "ES", "ET", "FI", "FJ", "FK", "FM", "FO", "FR", "GA", "GB", "GD", "GE", "GF", "GG", "GH", "GI", "GL", "GM", "GN", "GP", "GQ", "GR", "GS", "GT", "GU", "GW", "GY", "HK", "HM", "HN", "HR", "HT", "HU", "ID", "IE", "IL", "IM", "IN", "IO", "IQ", "IR", "IS", "IT", "JE", "JM", "JO", "JP", "KE", "KG", "KH", "KI", "KM", "KN", "KP", "KR", "KW", "KY", "KZ", "LA", "LB", "LC", "LI", "LK", "LR", "LS", "LT", "LU", "LV", "LY", "MA", "MC", "MD", "ME", "MF", "MG", "MH", "MK", "ML", "MM", "MN", "MO", "MP", "MQ", "MR", "MS", "MT", "MU", "MV", "MW", "MX", "MY", "MZ", "NA", "NC", "NE", "NF", "NG", "NI", "NL", "NO", "NP", "NR", "NU", "NZ", "OM", "PA", "PE", "PF", "PG", "PH", "PK", "PL", "PM", "PN", "PR", "PS", "PT", "PW", "PY", "QA", "RE", "RO", "RS", "RU", "RW", "SA", "SB", "SC", "SD", "SE", "SG", "SH", "SI", "SJ", "SK", "SL", "SM", "SN", "SO", "SR", "SS", "ST", "SV", "SX", "SY", "SZ", "TC", "TD", "TF", "TG", "TH", "TJ", "TK", "TL", "TM", "TN", "TO", "TR", "TT", "TV", "TW", "TZ", "UA", "UG", "UM", "US", "UY", "UZ", "VA", "VC", "VE", "VG", "VI", "VN", "VU", "WF", "WS", "YE", "YT", "ZA", "ZM", "ZW"}
 BYPASS_REGIONS = {
-  "UK",
+  "GB",
   "DE",
   "FR",
   "IN",
@@ -129,7 +129,6 @@ BYPASS_REGIONS = {
   "ID",
   "BD",
   "MX",
-  "ET",
   "PH",
   "EG",
   "VN",
@@ -274,6 +273,12 @@ class Video
     streams.each { |s| s.add("label", "#{s["quality"]} - #{s["type"].split(";")[0].split("/")[1]}") }
     streams = streams.uniq { |s| s["label"] }
 
+    if self.info["region"]?
+      streams.each do |fmt|
+        fmt["url"] += "&region=" + self.info["region"]
+      end
+    end
+
     if streams[0]? && streams[0]["s"]?
       streams.each do |fmt|
         fmt["url"] += "&signature=" + decrypt_signature(fmt["s"], decrypt_function)
@@ -363,6 +368,12 @@ class Video
       end
     end
 
+    if self.info["region"]?
+      adaptive_fmts.each do |fmt|
+        fmt["url"] += "&region=" + self.info["region"]
+      end
+    end
+
     if adaptive_fmts[0]? && adaptive_fmts[0]["s"]?
       adaptive_fmts.each do |fmt|
         fmt["url"] += "&signature=" + decrypt_signature(fmt["s"], decrypt_function)
@@ -466,14 +477,17 @@ class CaptionName
   )
 end
 
-def get_video(id, db, refresh = true)
+class VideoRedirect < Exception
+end
+
+def get_video(id, db, proxies = {} of String => Array({ip: String, port: Int32}), refresh = true)
   if db.query_one?("SELECT EXISTS (SELECT true FROM videos WHERE id = $1)", id, as: Bool)
     video = db.query_one("SELECT * FROM videos WHERE id = $1", id, as: Video)
 
     # If record was last updated over 10 minutes ago, refresh (expire param in response lasts for 6 hours)
     if refresh && Time.now - video.updated > 10.minutes
       begin
-        video = fetch_video(id)
+        video = fetch_video(id, proxies)
         video_array = video.to_a
 
         args = arg_array(video_array[1..-1], 2)
@@ -488,7 +502,7 @@ def get_video(id, db, refresh = true)
       end
     end
   else
-    video = fetch_video(id)
+    video = fetch_video(id, proxies)
     video_array = video.to_a
 
     args = arg_array(video_array)
@@ -499,15 +513,19 @@ def get_video(id, db, refresh = true)
   return video
 end
 
-def fetch_video(id)
-  html_channel = Channel(XML::Node).new
+def fetch_video(id, proxies)
+  html_channel = Channel(XML::Node | String).new
   info_channel = Channel(HTTP::Params).new
 
   spawn do
     client = make_client(YT_URL)
-    html = client.get("/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&disable_polymer=1")
-    html = XML.parse_html(html.body)
+    html = client.get("/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en&disable_polymer=1")
 
+    if md = html.headers["location"]?.try &.match(/v=(?<id>[a-zA-Z0-9_-]{11})/)
+      next html_channel.send(md["id"])
+    end
+
+    html = XML.parse_html(html.body)
     html_channel.send(html)
   end
 
@@ -525,48 +543,75 @@ def fetch_video(id)
   end
 
   html = html_channel.receive
+  if html.as?(String)
+    raise VideoRedirect.new("#{html.as(String)}")
+  end
+  html = html.as(XML::Node)
+
   info = info_channel.receive
 
   if info["reason"]? && info["reason"].includes? "your country"
-    bypass_channel = Channel({HTTP::Params | Nil, XML::Node | Nil}).new
+    bypass_channel = Channel(HTTPProxy | Nil).new
 
-    BYPASS_REGIONS.each do |country_code|
+    proxies.each do |region, list|
       spawn do
-        begin
-          proxies = get_proxies(country_code)
+        info = HTTP::Params.new({
+          "reason" => [info["reason"]],
+        })
 
-          # Try not to overload single proxy
-          proxy = proxies[0, 5].sample(1)[0]
-          proxy = HTTPProxy.new(proxy_host: proxy[:ip], proxy_port: proxy[:port])
+        list.each do |proxy|
+          begin
+            client = HTTPClient.new(YT_URL)
+            client.read_timeout = 10.seconds
+            client.connect_timeout = 10.seconds
 
-          client = HTTPClient.new(URI.parse("https://www.youtube.com"))
-          client.read_timeout = 10.seconds
-          client.connect_timeout = 10.seconds
-          client.set_proxy(proxy)
+            proxy = HTTPProxy.new(proxy_host: proxy[:ip], proxy_port: proxy[:port])
+            client.set_proxy(proxy)
 
-          proxy_info = client.get("/get_video_info?video_id=#{id}&el=detailpage&ps=default&eurl=&gl=US&hl=en&disable_polymer=1")
-          proxy_info = HTTP::Params.parse(proxy_info.body)
+            info = HTTP::Params.parse(client.get("/get_video_info?video_id=#{id}&ps=default&eurl=&gl=US&hl=en&disable_polymer=1").body)
+            if !info["reason"]?
+              bypass_channel.send(proxy)
+            else
+              bypass_channel.send(nil)
+            end
 
-          if !proxy_info["reason"]?
-            proxy_html = client.get("/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en&disable_polymer=1")
-            proxy_html = XML.parse_html(proxy_html.body)
-
-            bypass_channel.send({proxy_info, proxy_html})
-          else
-            bypass_channel.send({nil, nil})
+            break
+          rescue ex
           end
-        rescue ex
-          bypass_channel.send({nil, nil})
+        end
+
+        # If none of the proxies we tried returned a valid response
+        if info["reason"]?
+          bypass_channel.send(nil)
         end
       end
     end
 
-    BYPASS_REGIONS.size.times do
-      response = bypass_channel.receive
-      if response[0] || response[1]
-        info = response[0].not_nil!
-        html = response[1].not_nil!
-        break
+    proxies.size.times do
+      proxy = bypass_channel.receive
+      if proxy
+        begin
+          client = HTTPClient.new(YT_URL)
+          client.read_timeout = 10.seconds
+          client.connect_timeout = 10.seconds
+          client.set_proxy(proxy)
+
+          html = XML.parse_html(client.get("/watch?v=#{id}&bpctr=#{Time.new.epoch + 2000}&gl=US&hl=en&disable_polymer=1").body)
+          info = HTTP::Params.parse(client.get("/get_video_info?video_id=#{id}&el=detailpage&ps=default&eurl=&gl=US&hl=en&disable_polymer=1").body)
+
+          if info["reason"]?
+            info = HTTP::Params.parse(client.get("/get_video_info?video_id=#{id}&ps=default&eurl=&gl=US&hl=en&disable_polymer=1").body)
+          end
+
+          proxy = {ip: proxy.proxy_host, port: proxy.proxy_port}
+          region = proxies.select { |region, list| list.includes? proxy }
+          if !region.empty?
+            info["region"] = region.keys[0]
+          end
+
+          break
+        rescue ex
+        end
       end
     end
   end
@@ -603,10 +648,15 @@ def fetch_video(id)
 
   genre = html.xpath_node(%q(//meta[@itemprop="genre"])).not_nil!["content"]
   genre_url = html.xpath_node(%(//a[text()="#{genre}"])).try &.["href"]
-  if genre == "Movies"
-    genre_url ||= "/channel/UClgRkhTL3_hImCAmdLfDE4g"
+  case genre
+  when "Movies"
+    genre_url = "/channel/UClgRkhTL3_hImCAmdLfDE4g"
+  when "Education"
+    # Education channel is linked but does not exist
+    # genre_url = "/channel/UC3yA8nDwraeOfnYfBWun83g"
+    genre_url = ""
   end
-  genre_url = ""
+  genre_url ||= ""
 
   license = html.xpath_node(%q(//h4[contains(text(),"License")]/parent::*/ul/li))
   if license

src/invidious/views/components/item.ecr

@@ -14,7 +14,12 @@
         <p><%= number_with_separator(item.subscriber_count) %> subscribers</p>
         <h5><%= item.description_html %></h5>
     <% when SearchPlaylist %>
-        <a style="width:100%;" href="/playlist?list=<%= item.id %>">
+        <% if item.id.starts_with? "RD" %>
+        <% url = "/mix?list=#{item.id}&continuation=#{item.videos[0]?.try &.id}" %>
+        <% else %>
+        <% url = "/playlist?list=#{item.id}" %>
+        <% end %>
+        <a style="width:100%;" href="<%= url %>">
             <% if env.get?("user") && env.get("user").as(User).preferences.thin_mode %>
             <% else %>
             <img style="width:100%;" src="/vi/<%= item.videos[0]?.try &.id %>/mqdefault.jpg"/>
@@ -26,6 +31,17 @@
         </p>
         <p><%= number_with_separator(item.video_count) %> videos</p>
         <p>PLAYLIST</p>
+    <% when MixVideo %>
+        <a style="width:100%;" href="/watch?v=<%= item.id %>&list=<%= item.mixes[0] %>">
+            <% if env.get?("user") && env.get("user").as(User).preferences.thin_mode %>
+            <% else %>
+            <img style="width:100%;" src="/vi/<%= item.id %>/mqdefault.jpg"/>
+            <% end %>
+            <p><%= item.title %></p>
+        </a>
+        <p>
+            <b><a style="width:100%;" href="/channel/<%= item.ucid %>"><%= item.author %></a></b>
+        </p>
     <% else %>
         <% if item.responds_to?(:playlists) && !item.playlists.empty? %>
         <% params = "&list=#{item.playlists[0]}" %>

src/invidious/views/data_control.ecr

@@ -13,7 +13,7 @@
             </div>
 
             <div class="pure-control-group">
-                <label for="import_youtube">Import <a target="_blank"
+                <label for="import_youtube">Import <a rel="noopener" target="_blank"
                         href="https://support.google.com/youtube/answer/6224202?hl=en-GB">YouTube subscriptions</a></label>
                 <input type="file" id="import_youtube" name="import_youtube">
             </div>

src/invidious/views/mix.ecr

@@ -0,0 +1,22 @@
+<% content_for "header" do %>
+<title><%= mix.title %> - Invidious</title>
+<% end %>
+
+<div class="pure-g h-box">
+    <div class="pure-u-2-3">
+        <h3><%= mix.title %></h3>
+    </div>
+    <div class="pure-u-1-3" style="text-align:right;">
+        <h3>
+            <a href="/feed/playlist/<%= mix.id %>"><i class="icon ion-logo-rss"></i></a>
+        </h3>
+    </div>
+</div>
+
+<% mix.videos.each_slice(4) do |slice| %>
+<div class="pure-g">
+    <% slice.each do |item| %>
+        <%= rendered "components/item" %>
+    <% end %>
+</div> 
+<% end %>

src/invidious/views/playlist.ecr

@@ -35,7 +35,7 @@
 <div class="pure-g h-box">
     <div class="pure-u-1 pure-u-md-1-5">
     <% if page >= 2 %>
-        <a href="/playlist?list=<%= playlist.id %>&page=<%= page - 1 %>">Next page</a>
+        <a href="/playlist?list=<%= playlist.id %>&page=<%= page - 1 %>">Previous page</a>
     <% end %>
     </div>
     <div class="pure-u-1 pure-u-md-3-5"></div>

src/invidious/views/search.ecr

@@ -13,13 +13,13 @@
 <div class="pure-g h-box">
     <div class="pure-u-1 pure-u-md-1-5">
     <% if page >= 2 %>
-        <a href="/search?q=<%= query %>&page=<%= page - 1 %>">Previous page</a>
+        <a href="/search?q=<%= HTML.escape(query.not_nil!) %>&page=<%= page - 1 %>">Previous page</a>
     <% end %>
     </div>
     <div class="pure-u-1 pure-u-md-3-5"></div>
     <div style="text-align:right;" class="pure-u-1 pure-u-md-1-5">
     <% if count >= 20 %>
-        <a href="/search?q=<%= query %>&page=<%= page + 1 %>">Next page</a>
+        <a href="/search?q=<%= HTML.escape(query.not_nil!) %>&page=<%= page + 1 %>">Next page</a>
     <% end %>
     </div>
 </div>

src/invidious/views/template.ecr

@@ -28,7 +28,7 @@
         <div class="pure-u-1 pure-u-md-12-24 searchbar">
           <form class="pure-form" action="/search" method="get">
             <fieldset>
-              <input type="search" style="width:100%;" name="q" placeholder="search" value="<%= env.params.query["q"]? || env.get? "search" %>">
+              <input type="search" style="width:100%;" name="q" placeholder="search" value="<%= env.params.query["q"]?.try {|x| HTML.escape(x)} || env.get?("search").try {|x| HTML.escape(x.as(String)) }  %>">
             </fieldset>
           </form>
         </div>

src/invidious/views/watch.ecr

@@ -22,6 +22,7 @@
 <meta name="twitter:player" content="<%= host_url %>/embed/<%= video.id %>">
 <meta name="twitter:player:width" content="1280">
 <meta name="twitter:player:height" content="720">
+<script src="/js/watch.js"></script>
 <%= rendered "components/player_sources" %>
 <title><%= HTML.escape(video.title) %> - Invidious</title>
 <% end %>
@@ -55,7 +56,13 @@
             <p><i class="icon ion-ios-eye"></i> <%= number_with_separator(video.views) %></p>
             <p><i class="icon ion-ios-thumbs-up"></i> <%= number_with_separator(video.likes) %></p>
             <p><i class="icon ion-ios-thumbs-down"></i> <%= number_with_separator(video.dislikes) %></p>
-            <p id="Genre">Genre: <a href="<%= video.genre_url %>"><%= video.genre %></a></p>
+            <p id="Genre">Genre: 
+            <% if video.genre_url.empty? %>
+            <%= video.genre %>
+            <% else %>
+            <a href="<%= video.genre_url %>"><%= video.genre %></a>
+            <% end %>
+            </p>
             <% if !video.license.empty? %>
             <p id="License">License: <%= video.license %></p>
             <% end %>
@@ -116,6 +123,13 @@
         </div>
     </div>
     <div class="pure-u-1 pure-u-md-1-5">
+        <% if plid %>
+        <div id="playlist" class="h-box">
+            <h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>
+            <hr>
+        </div>
+        <% end %>
+
         <% if !preferences || preferences && preferences.related_videos %>
         <div class="h-box">
         <% rvs.each do |rv| %>
@@ -138,61 +152,61 @@
 </div>
 
 <script>
-function toggle(target) {
-  body = target.parentNode.parentNode.children[1];
-  if (body.style.display === null || body.style.display === "") {
-    target.innerHTML = "[ + ]";
-    body.style.display = "none";
-  } else {
-    target.innerHTML = "[ - ]";
-    body.style.display = "";
-  }
-}
+<% if plid %>
+function get_playlist() {
+    var plid = "<%= plid %>"
 
-function toggle_comments(target) {
-  body = target.parentNode.parentNode.parentNode.children[1];
-  if (body.style.display === null || body.style.display === "") {
-    target.innerHTML = "[ + ]";
-    body.style.display = "none";
-  } else {
-    target.innerHTML = "[ - ]";
-    body.style.display = "";
-  }
-}
-
-function get_youtube_replies(target) {
-  var continuation = target.getAttribute("data-continuation");
-
-  var body = target.parentNode.parentNode;
-  var fallback = body.innerHTML;
-  body.innerHTML =
-    '<h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>';
-
-  var url =
-    "/api/v1/comments/<%= video.id %>?format=html&continuation=" + continuation;
-  var xhr = new XMLHttpRequest();
-  xhr.responseType = "json";
-  xhr.timeout = 20000;
-  xhr.open("GET", url, true);
-  xhr.send();
-
-  xhr.onreadystatechange = function() {
-    if (xhr.readyState == 4) {
-      if (xhr.status == 200) {
-        body.innerHTML = xhr.response.contentHtml;
-      } else {
-        body.innerHTML = fallback;
-      }
+    if (plid.startsWith("RD")) {
+        var plid_url = "/api/v1/mixes/<%= plid %>?continuation=<%= video.id %>&format=html";
+    } else {
+        var plid_url = "/api/v1/playlists/<%= plid %>?continuation=<%= video.id %>&format=html";
     }
-  };
 
-  xhr.ontimeout = function() {
-    console.log("Pulling comments timed out.");
+    var xhr = new XMLHttpRequest();
+    xhr.responseType = "json";
+    xhr.timeout = 20000;
+    xhr.open("GET", plid_url, true);
+    xhr.send();
 
-    body.innerHTML = fallback;
-  };
+    xhr.onreadystatechange = function() {
+        if (xhr.readyState == 4) {
+            if (xhr.status == 200) {
+                playlist = document.getElementById("playlist");
+                playlist.innerHTML = xhr.response.playlistHtml;
+                
+                if (xhr.response.nextVideo) {
+                    player.on('ended', function() {
+                        window.location.replace("/watch?v=" 
+                            + xhr.response.nextVideo
+                            + "&list=<%= plid %>"
+                            <% if params[:listen] %>
+                            + "&listen=1"
+                            <% end %>
+                            <% if params[:autoplay] %>
+                            + "&autoplay=1"
+                            <% end %>
+                        );
+                    });
+                }
+            } else {
+                playlist.innerHTML = "";
+            }
+        }
+    };
+
+    xhr.ontimeout = function() {
+        console.log("Pulling playlist timed out.");
+
+        comments = document.getElementById("playlist");
+        comments.innerHTML =
+            '<h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3><hr>';
+        get_playlist();
+    };
 }
 
+get_playlist();
+<% end %>
+
 function get_reddit_comments() {
   var url = "/api/v1/comments/<%= video.id %>?source=reddit&format=html";
   var xhr = new XMLHttpRequest();
@@ -202,7 +216,7 @@ function get_reddit_comments() {
   xhr.send();
 
   xhr.onreadystatechange = function() {
-    if (xhr.readyState == 4)
+    if (xhr.readyState == 4) {
       if (xhr.status == 200) {
         comments = document.getElementById("comments");
         comments.innerHTML = ' \
@@ -211,8 +225,15 @@ function get_reddit_comments() {
                     <a href="javascript:void(0)" onclick="toggle_comments(this)">[ - ]</a> \
                     {title} \
                 </h3> \
+                <p> \
+                    <b> \
+                        <a href="javascript:void(0)" onclick="swap_comments(\'youtube\')"> \
+                            View YouTube comments \
+                        </a> \
+                    </b> \
+                </p> \
                 <b> \
-                    <a target="_blank" href="https://reddit.com{permalink}">View more comments on Reddit</a> \
+                    <a rel="noopener" target="_blank" href="https://reddit.com{permalink}">View more comments on Reddit</a> \
                 </b> \
             </div> \
             <div>{contentHtml}</div> \
@@ -229,6 +250,7 @@ function get_reddit_comments() {
         comments.innerHTML = "";
         <% end %>
       }
+    }
   };
 
   xhr.ontimeout = function() {
@@ -247,7 +269,7 @@ function get_youtube_comments() {
   xhr.send();
 
   xhr.onreadystatechange = function() {
-    if (xhr.readyState == 4)
+    if (xhr.readyState == 4) {
       if (xhr.status == 200) {
         comments = document.getElementById("comments");
         if (xhr.response.commentCount > 0)  {
@@ -257,6 +279,11 @@ function get_youtube_comments() {
                         <a href="javascript:void(0)" onclick="toggle_comments(this)">[ - ]</a> \
                         View {commentCount} comments \
                     </h3> \
+                    <b> \
+                        <a href="javascript:void(0)" onclick="swap_comments(\'reddit\')"> \
+                            View Reddit comments \
+                        </a> \
+                    </b> \
                 </div> \
                 <div>{contentHtml}</div> \
                 <hr>'.supplant({
@@ -274,6 +301,7 @@ function get_youtube_comments() {
         comments.innerHTML = "";
         <% end %>
       }
+    }
   };
 
   xhr.ontimeout = function() {
@@ -286,19 +314,38 @@ function get_youtube_comments() {
   };
 }
 
-function commaSeparateNumber(val){
-  while (/(\d+)(\d{3})/.test(val.toString())){
-    val = val.toString().replace(/(\d+)(\d{3})/, '$1'+','+'$2');
-  }
-  return val;
-}
+function get_youtube_replies(target) {
+  var continuation = target.getAttribute('data-continuation');
 
-String.prototype.supplant = function(o) {
-  return this.replace(/{([^{}]*)}/g, function(a, b) {
-    var r = o[b];
-    return typeof r === "string" || typeof r === "number" ? r : a;
-  });
-};
+  var body = target.parentNode.parentNode;
+  var fallback = body.innerHTML;
+  body.innerHTML =
+      '<h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>';
+
+  var url = '/api/v1/comments/<%= video.id %>?format=html&continuation=' +
+      continuation;
+  var xhr = new XMLHttpRequest();
+  xhr.responseType = 'json';
+  xhr.timeout = 20000;
+  xhr.open('GET', url, true);
+  xhr.send();
+
+  xhr.onreadystatechange = function() {
+    if (xhr.readyState == 4) {
+      if (xhr.status == 200) {
+        body.innerHTML = xhr.response.contentHtml;
+      } else {
+        body.innerHTML = fallback;
+      }
+    }
+  };
+
+  xhr.ontimeout = function() {
+    console.log('Pulling comments timed out.');
+
+    body.innerHTML = fallback;
+  };
+}
 
 <% if preferences %>
     <% if preferences.comments[0] == "youtube" %>