Don't nest YouTube replies

This reverts commit 95c6747a3e.

CHANGELOG.md

@@ -1,3 +1,36 @@
+# 0.10.0 (2018-10-16)
+
+## Week 10: Subscriptions
+
+This week I'm happy to announce that subscriptions have been drastically sped up with
+35e63fa. As I mentioned last week, this essentially "caches" a user's feed, meaning that operations that previously took 20 seconds or timed out, now can load in under a second. I'd take a look at [#173](https://github.com/omarroth/invidious/issues/173) for a sample benchmark. Previously features that made Invidious's feed so useful, such as filtering by unseen and by author would take too long to load, and so instead would timeout. I'm very happy that this has been fixed, and folks can get back to using these features.
+
+Among some smaller features that have been added this week include [#118](https://github.com/omarroth/invidious/issues/118), which adds, in my opinion, some very attractive subscribe and unsubscribe buttons. I think it's also a bit of a functional improvement as well, since it doesn't require a user to reload the page in order to subscribe or unsubscribe to a channel, and also gives the opportunity to put the channel's sub count on display.
+
+An option to swap between Reddit and YouTube comments without a page reload has been added with
+5eefab6, bringing it somewhat closer in functionality to the popular [AlienTube](https://github.com/xlexi/alientube) extension, on which it is based (although the extension unfortunately appears now to be fragmented).
+
+As always, there are a couple smaller improvements this week, including some minor fixes for geo-bypass with
+e46e618 and [`245d0b5`](https://github.com/omarroth/invidious/245d0b5), playlist preferences with [`81b4477`](https://github.com/omarroth/invidious/81b4477), and YouTube comments with [`02335f3`](https://github.com/omarroth/invidious/02335f3).
+
+This coming week I'd also recommend keeping an eye on the excellent [FreeTube](https://github.com/FreeTubeApp/FreeTube), which is looking forward to a new release. I've been very lucky to work with [**@PrestonN**](https://github.com/PrestonN) for the past few weeks to improve the Invidious API, and I'm quite looking forward to the new release.
+
+That's all for this week folks, thank you all again for your continued interest and support.
+
+# 0.9.0 (2018-10-08)
+
+## Week 9: Playlists
+
+Not as much to announce this week, but I'm still quite happy to announce a couple things, namely:
+
+Playback support for playlists has finally been added with [`88430a6`](https://github.com/omarroth/invidious/88430a6). You can now view playlists with the `&list=` query param, as you would on YouTube. You can also view mixes with the mentioned `&list=`, although they require some extra handling that I would like to add in the coming week, as well as adding playlist looping and shuffle. I think playback support has been a roadblock for more exciting features such as [#114](https://github.com/omarroth/invidious/issues/114), and I look forward to improving the experience.
+
+Comments have had a bit of a cosmetic upgrade with [#132](https://github.com/omarroth/invidious/issues/132), which I think helps better distinguish between Reddit and YouTube comments, as it makes them appear similarly to their respective sites. You can also now switch between YouTube and Reddit comments with a push of a button, which I think is quite an improvement, especially for newer or less popular videos with fewer comments.
+
+I've had a small breakthrough in speeding up users' subscription feeds with PostgreSQL's [materialized views](https://www.postgresql.org/docs/current/static/rules-materializedviews.html). Without going into too much detail, materialized views essentially cache the result of a query, making it possible to run resource-intensive queries once, rather than every time a user visits their feed. In the coming week I hope to push this out to users, and hopefully close [#173](https://github.com/omarroth/invidious/issues/173).
+
+I haven't had as much time to work on the project this week, but I'm quite happy to have added some new features. Have a great week everyone.
+
 # 0.8.0 (2018-10-02)
 
 ## Week 8: Mixes
@@ -29,6 +62,7 @@ A [CHANGELOG](https://github.com/omarroth/invidious/blob/master/CHANGELOG.md) ha
 Recently, users have been reporting 504s when attempting to access their subscriptions, which is tracked in [#173](https://github.com/omarroth/invidious/issues/173). This is most likely caused by an uptick in usage, which I am absolutely grateful for, but unfortunately has resulted in an increase in costs for hosting the site, which is why I will be bumping my goal on Patreon from $60 to $80. I would appreciate any feedback on how subscriptions could be improved.
 
 Other minor improvements include:
+
 - Additional regions added to bypass geo-block with [`9a78523`](https://github.com/omarroth/invidious/9a78523)
 - Fix for playlists containing less than 100 videos (previously shown as empty) with [`35ac887`](https://github.com/omarroth/invidious/35ac887)
 - Fix for `published` date for Reddit comments (previously showing negative seconds) with [`6e09202`](https://github.com/omarroth/invidious/6e09202)
@@ -41,7 +75,7 @@ Thank you everyone for your support!
 
 Hello again! This week I'm happy to mention a couple new features to search as well as some miscellaneous usability improvements.
 
-You can now constrain your search query to a specific channel with the `channel:CHANNEL` filter (see [#165](https://github.com/omarroth/invidious/issues/165) for more details). Unfortunately, other search filters combined with channel search are not yet supported. I hope to add support for them in the coming weeks. 
+You can now constrain your search query to a specific channel with the `channel:CHANNEL` filter (see [#165](https://github.com/omarroth/invidious/issues/165) for more details). Unfortunately, other search filters combined with channel search are not yet supported. I hope to add support for them in the coming weeks.
 
 You can also now search only your subscriptions by adding `subscriptions:true` to your query (see [#30](https://github.com/omarroth/invidious/issues/30) for more details). It's not quite ready for widespread use but I would appreciate feedback as the site updates to fully support it. Other search filters are not yet supported with `subscriptions:true`, but I hope to add more functionality to this as well.
 
@@ -67,7 +101,7 @@ A potential XSS vector has also been fixed in YouTube comments with [`8c45694`](
 
 All the above vulnerabilities were brought to my attention by someone who wishes to remain anonymous, but I would like to say again here how thankful I am. If anyone else would like to get in touch please feel free to email me at omarroth@hotmail.com or omarroth@protonmail.com.
 
-This week a couple changes have been made to better protect user's privacy as well. 
+This week a couple changes have been made to better protect user's privacy as well.
 All CSS and JS assets are now served locally with [`3ec684a`](https://github.com/omarroth/invidious/3ec684a), which means users no longer need to whitelist unpkg.com. Although I personally have encountered few issues, I understand that many folks would like to keep their browsing activity contained to as few parties as possible. In the coming week I also hope to proxy YouTube images, so that no user data is sent to Google.
 
 YouTube links in comments now should redirect properly to the Invidious alternate with [`1c8bd67`](https://github.com/omarroth/invidious/1c8bd67) and [`cf63c82`](https://github.com/omarroth/invidious/cf63c82), so users can more easily evade Google tracking.
@@ -117,7 +151,7 @@ I'd also like to announce that I've set up an account on [Liberapay](https://lib
 
 [Two weeks ago](https://github.com/omarroth/invidious/releases/tag/0.1.0) I mentioned adding 1080p support into the player. Currently, the only thing blocking is [#207](https://github.com/videojs/http-streaming/pull/207) in the excellent [http-streaming](https://github.com/videojs/http-streaming) library. I hope to work with the videojs team to merge it soon and finally implement 1080p support!
 
-That's all for this week, thank you again everyone for your support! 
+That's all for this week, thank you again everyone for your support!
 
 # 0.2.0 (2018-09-06)
 
@@ -132,9 +166,9 @@ A couple of miscellaneous features and bugfixes:
 
 - You can now login to Invidious simultaneously from multiple devices - [#109](https://github.com/omarroth/invidious/issues/109)
 
-- Added a note for scheduled livestreams - [#124](https://github.com/omarroth/invidious/issues/124) 
+- Added a note for scheduled livestreams - [#124](https://github.com/omarroth/invidious/issues/124)
 
-- Changed YouTube comment header to "View x comments" - [#120](https://github.com/omarroth/invidious/issues/120) 
+- Changed YouTube comment header to "View x comments" - [#120](https://github.com/omarroth/invidious/issues/120)
 
 Enjoy your week everyone!
 

assets/css/default.css

@@ -22,6 +22,39 @@ div {
   padding-right: 10px;
 }
 
+a.pure-button-primary {
+  background-color: #a0a0a0;
+  color: rgba(35, 35, 35, 1);
+}
+
+a.pure-button-primary:hover {
+  background-color: rgba(0, 182, 240, 1);
+  color: #fff;
+}
+
+div.thumbnail {
+  position: relative;
+}
+
+img.thumbnail {
+  width: 100%;
+  left: 0;
+  top: 0;
+}
+
+.length {
+  z-index: 100;
+  position: absolute;
+  background-color: rgba(35, 35, 35, 0.75);
+  color: #fff;
+  border-radius: 2px;
+  padding: 2px;
+  font-size: 16px;
+  font-family: sans-serif;
+  right: 0.5em;
+  bottom: -0.5em;
+}
+
 /*
  * Navbar
  */

assets/js/watch.js

@@ -21,11 +21,6 @@ function toggle_comments(target) {
 }
 
 function swap_comments(source) {
-  comments = document.getElementById("comments");
-  var fallback = comments.innerHTML;
-  comments.innerHTML =
-    '<h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>';
-
   if (source == "youtube") {
     get_youtube_comments();
   } else if (source == "reddit") {
@@ -33,16 +28,25 @@ function swap_comments(source) {
   }
 }
 
-function commaSeparateNumber(val) {
-  while (/(\d+)(\d{3})/.test(val.toString())) {
-    val = val.toString().replace(/(\d+)(\d{3})/, "$1" + "," + "$2");
-  }
-  return val;
-}
-
 String.prototype.supplant = function(o) {
   return this.replace(/{([^{}]*)}/g, function(a, b) {
     var r = o[b];
     return typeof r === "string" || typeof r === "number" ? r : a;
   });
 };
+
+function show_youtube_replies(target) {
+  body = target.parentNode.parentNode.children[1];
+  body.style.display = "";
+
+  target.innerHTML = "Hide replies";
+  target.setAttribute("onclick", "hide_youtube_replies(this)");
+}
+
+function hide_youtube_replies(target) {
+  body = target.parentNode.parentNode.children[1];
+  body.style.display = "none";
+
+  target.innerHTML = "Show replies";
+  target.setAttribute("onclick", "show_youtube_replies(this)");
+}

config/config.yml

@@ -1,5 +1,6 @@
 crawl_threads: 1
 channel_threads: 1
+feed_threads: 1
 video_threads: 1
 db:
   user: kemal

config/sql/videos.sql

@@ -22,6 +22,8 @@ CREATE TABLE public.videos
     genre text COLLATE pg_catalog."default",
     genre_url text COLLATE pg_catalog."default",
     license text COLLATE pg_catalog."default",
+    sub_count_text text COLLATE pg_catalog."default",
+    author_thumbnail text COLLATE pg_catalog."default",
     CONSTRAINT videos_pkey PRIMARY KEY (id)
 )
 WITH (

shard.yml

@@ -1,5 +1,5 @@
 name: invidious
-version: 0.8.0
+version: 0.10.0
 
 authors:
   - Omar Roth <omarroth@hotmail.com>

src/invidious.cr

@@ -31,6 +31,7 @@ HMAC_KEY = CONFIG.hmac_key || Random::Secure.random_bytes(32)
 
 crawl_threads = CONFIG.crawl_threads
 channel_threads = CONFIG.channel_threads
+feed_threads = CONFIG.feed_threads
 video_threads = CONFIG.video_threads
 
 Kemal.config.extra_options do |parser|
@@ -51,6 +52,14 @@ Kemal.config.extra_options do |parser|
       exit
     end
   end
+  parser.on("-f THREADS", "--feed-threads=THREADS", "Number of threads for refreshing feeds (default: #{feed_threads})") do |number|
+    begin
+      feed_threads = number.to_i
+    rescue ex
+      puts "THREADS must be integer"
+      exit
+    end
+  end
   parser.on("-v THREADS", "--video-threads=THREADS", "Number of threads for refreshing videos (default: #{video_threads})") do |number|
     begin
       video_threads = number.to_i
@@ -85,6 +94,8 @@ end
 
 refresh_channels(PG_DB, channel_threads, CONFIG.full_refresh)
 
+refresh_feeds(PG_DB, feed_threads)
+
 video_threads.times do |i|
   spawn do
     refresh_videos(PG_DB)
@@ -475,9 +486,8 @@ get "/search" do |env|
   user = env.get? "user"
   if user
     user = user.as(User)
-    ucids = user.subscriptions
+    view_name = "subscriptions_#{sha256(user.email)[0..7]}"
   end
-  ucids ||= [] of String
 
   channel = nil
   content_type = "all"
@@ -514,14 +524,19 @@ get "/search" do |env|
   if channel
     count, videos = channel_search(search_query, page, channel)
   elsif subscriptions
-    videos = PG_DB.query_all("SELECT id,title,published,updated,ucid,author FROM (
+    if view_name
+      videos = PG_DB.query_all("SELECT id,title,published,updated,ucid,author FROM (
       SELECT *,
-      to_tsvector(channel_videos.title) ||
-      to_tsvector(channel_videos.author)
+      to_tsvector(#{view_name}.title) ||
+      to_tsvector(#{view_name}.author)
       as document
-      FROM channel_videos WHERE ucid IN (#{arg_array(ucids, 3)})
-      ) v_search WHERE v_search.document @@ plainto_tsquery($1) LIMIT 20 OFFSET $2;", [search_query, (page - 1) * 20] + ucids, as: ChannelVideo)
-    count = videos.size
+      FROM #{view_name}
+      ) v_search WHERE v_search.document @@ plainto_tsquery($1) LIMIT 20 OFFSET $2;", search_query, (page - 1) * 20, as: ChannelVideo)
+      count = videos.size
+    else
+      videos = [] of ChannelVideo
+      count = 0
+    end
   else
     begin
       search_params = produce_search_params(sort: sort, date: date, content_type: content_type,
@@ -755,7 +770,7 @@ post "/login" do |env|
     end
 
     if action == "signin"
-      user = PG_DB.query_one?("SELECT * FROM users WHERE email = $1 AND password IS NOT NULL", email, as: User)
+      user = PG_DB.query_one?("SELECT * FROM users WHERE LOWER(email) = LOWER($1) AND password IS NOT NULL", email, as: User)
 
       if !user
         error_message = "Invalid username or password"
@@ -769,7 +784,7 @@ post "/login" do |env|
 
       if Crypto::Bcrypt::Password.new(user.password.not_nil!) == password
         sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
-        PG_DB.exec("UPDATE users SET id = id || $1 WHERE email = $2", [sid], email)
+        PG_DB.exec("UPDATE users SET id = id || $1 WHERE LOWER(email) = LOWER($2)", [sid], email)
 
         if Kemal.config.ssl || CONFIG.https_only
           secure = true
@@ -784,7 +799,7 @@ post "/login" do |env|
         next templated "error"
       end
     elsif action == "register"
-      user = PG_DB.query_one?("SELECT * FROM users WHERE email = $1 AND password IS NOT NULL", email, as: User)
+      user = PG_DB.query_one?("SELECT * FROM users WHERE LOWER(email) = LOWER($1) AND password IS NOT NULL", email, as: User)
       if user
         error_message = "Please sign in"
         next templated "error"
@@ -799,6 +814,12 @@ post "/login" do |env|
 
       PG_DB.exec("INSERT INTO users VALUES (#{args})", user_array)
 
+      view_name = "subscriptions_#{sha256(user.email)[0..7]}"
+      PG_DB.exec("CREATE MATERIALIZED VIEW #{view_name} AS \
+        SELECT * FROM channel_videos WHERE \
+        ucid = ANY ((SELECT subscriptions FROM users WHERE email = '#{user.email}')::text[]) \
+      ORDER BY published DESC;")
+
       if Kemal.config.ssl || CONFIG.https_only
         secure = true
       else
@@ -1364,6 +1385,8 @@ get "/feed/subscriptions" do |env|
 
     notifications = PG_DB.query_one("SELECT notifications FROM users WHERE email = $1", user.email,
       as: Array(String))
+    view_name = "subscriptions_#{sha256(user.email)[0..7]}"
+
     if preferences.notifications_only && !notifications.empty?
       args = arg_array(notifications)
 
@@ -1386,39 +1409,35 @@ get "/feed/subscriptions" do |env|
     else
       if preferences.latest_only
         if preferences.unseen_only
-          ucids = arg_array(user.subscriptions)
           if user.watched.empty?
             watched = "'{}'"
           else
-            watched = arg_array(user.watched, user.subscriptions.size + 1)
+            watched = arg_array(user.watched)
           end
 
-          videos = PG_DB.query_all("SELECT DISTINCT ON (ucid) * FROM channel_videos WHERE \
-            ucid IN (#{ucids}) AND id NOT IN (#{watched}) ORDER BY ucid, published DESC",
-            user.subscriptions + user.watched, as: ChannelVideo)
+          videos = PG_DB.query_all("SELECT DISTINCT ON (ucid) * FROM #{view_name} WHERE \
+            id NOT IN (#{watched}) ORDER BY ucid, published DESC",
+            user.watched, as: ChannelVideo)
         else
-          args = arg_array(user.subscriptions)
-          videos = PG_DB.query_all("SELECT DISTINCT ON (ucid) * FROM channel_videos WHERE \
-          ucid IN (#{args}) ORDER BY ucid, published DESC", user.subscriptions, as: ChannelVideo)
+          videos = PG_DB.query_all("SELECT DISTINCT ON (ucid) * FROM #{view_name} \
+          ORDER BY ucid, published DESC", as: ChannelVideo)
         end
 
         videos.sort_by! { |video| video.published }.reverse!
       else
         if preferences.unseen_only
-          ucids = arg_array(user.subscriptions, 3)
           if user.watched.empty?
             watched = "'{}'"
           else
-            watched = arg_array(user.watched, user.subscriptions.size + 3)
+            watched = arg_array(user.watched, 3)
           end
 
-          videos = PG_DB.query_all("SELECT * FROM channel_videos WHERE ucid IN (#{ucids}) \
-          AND id NOT IN (#{watched}) ORDER BY published DESC LIMIT $1 OFFSET $2",
-            [limit, offset] + user.subscriptions + user.watched, as: ChannelVideo)
+          videos = PG_DB.query_all("SELECT * FROM #{view_name} WHERE \
+          id NOT IN (#{watched}) LIMIT $1 OFFSET $2",
+            [limit, offset] + user.watched, as: ChannelVideo)
         else
-          args = arg_array(user.subscriptions, 3)
-          videos = PG_DB.query_all("SELECT * FROM channel_videos WHERE ucid IN (#{args}) \
-          ORDER BY published DESC LIMIT $1 OFFSET $2", [limit, offset] + user.subscriptions, as: ChannelVideo)
+          videos = PG_DB.query_all("SELECT * FROM #{view_name} \
+          ORDER BY published DESC LIMIT $1 OFFSET $2", limit, offset, as: ChannelVideo)
         end
       end
 
@@ -1467,29 +1486,8 @@ get "/feed/channel/:ucid" do |env|
     halt env, status_code: 404, response: error_message
   end
 
-  client = make_client(YT_URL)
-
   page = 1
-
-  videos = [] of SearchVideo
-  2.times do |i|
-    url = produce_channel_videos_url(ucid, page * 2 + (i - 1), auto_generated: auto_generated)
-    response = client.get(url)
-    json = JSON.parse(response.body)
-
-    if json["content_html"]? && !json["content_html"].as_s.empty?
-      document = XML.parse_html(json["content_html"].as_s)
-      nodeset = document.xpath_nodes(%q(//li[contains(@class, "feed-item-container")]))
-
-      if auto_generated
-        videos += extract_videos(nodeset)
-      else
-        videos += extract_videos(nodeset, ucid)
-      end
-    else
-      break
-    end
-  end
+  videos, count = get_60_videos(ucid, page, auto_generated)
 
   host_url = make_host_url(Kemal.config.ssl || CONFIG.https_only, env.request.headers["Host"]?)
   path = env.request.path
@@ -1576,15 +1574,14 @@ get "/feed/private" do |env|
   latest_only ||= 0
   latest_only = latest_only == 1
 
+  view_name = "subscriptions_#{sha256(user.email)[0..7]}"
+
   if latest_only
-    args = arg_array(user.subscriptions)
-    videos = PG_DB.query_all("SELECT DISTINCT ON (ucid) * FROM channel_videos WHERE \
-    ucid IN (#{args}) ORDER BY ucid, published DESC", user.subscriptions, as: ChannelVideo)
+    videos = PG_DB.query_all("SELECT DISTINCT ON (ucid) * FROM #{view_name} ORDER BY ucid, published DESC", as: ChannelVideo)
     videos.sort_by! { |video| video.published }.reverse!
   else
-    args = arg_array(user.subscriptions, 3)
-    videos = PG_DB.query_all("SELECT * FROM channel_videos WHERE ucid IN (#{args}) \
-  ORDER BY published DESC LIMIT $1 OFFSET $2", [limit, offset] + user.subscriptions, as: ChannelVideo)
+    videos = PG_DB.query_all("SELECT * FROM #{view_name} \
+    ORDER BY published DESC LIMIT $1 OFFSET $2", limit, offset, as: ChannelVideo)
   end
 
   sort = env.params.query["sort"]?
@@ -1721,7 +1718,7 @@ get "/channel/:ucid" do |env|
   page ||= 1
 
   begin
-    author, ucid, auto_generated = get_about_info(ucid)
+    author, ucid, auto_generated, sub_count = get_about_info(ucid)
   rescue ex
     error_message = "User does not exist"
     next templated "error"
@@ -1735,27 +1732,7 @@ get "/channel/:ucid" do |env|
     end
   end
 
-  client = make_client(YT_URL)
-
-  videos = [] of SearchVideo
-  2.times do |i|
-    url = produce_channel_videos_url(ucid, page * 2 + (i - 1), auto_generated: auto_generated)
-    response = client.get(url)
-    json = JSON.parse(response.body)
-
-    if json["content_html"]? && !json["content_html"].as_s.empty?
-      document = XML.parse_html(json["content_html"].as_s)
-      nodeset = document.xpath_nodes(%q(//li[contains(@class, "feed-item-container")]))
-
-      if auto_generated
-        videos += extract_videos(nodeset)
-      else
-        videos += extract_videos(nodeset, ucid)
-      end
-    else
-      break
-    end
-  end
+  videos, count = get_60_videos(ucid, page, auto_generated)
 
   templated "channel"
 end
@@ -1908,7 +1885,6 @@ get "/api/v1/comments/:id" do |env|
               proxy_client.read_timeout = 10.seconds
               proxy_client.connect_timeout = 10.seconds
 
-              proxy = list.sample(1)[0]
               proxy = HTTPProxy.new(proxy_host: proxy[:ip], proxy_port: proxy[:port])
               proxy_client.set_proxy(proxy)
 
@@ -1917,13 +1893,10 @@ get "/api/v1/comments/:id" do |env|
               proxy_headers["cookie"] = response.cookies.add_request_headers(headers)["cookie"]
               proxy_html = response.body
 
-              if proxy_html.match(/<meta itemprop="regionsAllowed" content="">/)
-                bypass_channel.send(nil)
-              else
+              if !proxy_html.match(/<meta itemprop="regionsAllowed" content="">/)
                 bypass_channel.send({proxy_html, proxy_client, proxy_headers})
+                break
               end
-
-              break
             rescue ex
             end
           end
@@ -2272,6 +2245,8 @@ get "/api/v1/videos/:id" do |env|
       json.field "likeCount", video.likes
       json.field "dislikeCount", video.dislikes
 
+      json.field "paid", video.paid
+      json.field "premium", video.premium
       json.field "isFamilyFriendly", video.is_family_friendly
       json.field "allowedRegions", video.allowed_regions
       json.field "genre", video.genre
@@ -2281,6 +2256,22 @@ get "/api/v1/videos/:id" do |env|
       json.field "authorId", video.ucid
       json.field "authorUrl", "/channel/#{video.ucid}"
 
+      json.field "authorThumbnails" do
+        json.array do
+          qualities = [32, 48, 76, 100, 176, 512]
+
+          qualities.each do |quality|
+            json.object do
+              json.field "url", video.author_thumbnail.gsub("=s48-", "=s#{quality}-")
+              json.field "width", quality
+              json.field "height", quality
+            end
+          end
+        end
+      end
+
+      json.field "subCountText", video.sub_count_text
+
       json.field "lengthSeconds", video.info["length_seconds"].to_i
       if video.info["allow_ratings"]?
         json.field "allowRatings", video.info["allow_ratings"] == "1"
@@ -2499,30 +2490,10 @@ get "/api/v1/channels/:ucid" do |env|
     halt env, status_code: 404, response: error_message
   end
 
-  client = make_client(YT_URL)
-
   page = 1
+  videos, count = get_60_videos(ucid, page, auto_generated)
 
-  videos = [] of SearchVideo
-  2.times do |i|
-    url = produce_channel_videos_url(ucid, page * 2 + (i - 1), auto_generated: auto_generated)
-    response = client.get(url)
-    json = JSON.parse(response.body)
-
-    if json["content_html"]? && !json["content_html"].as_s.empty?
-      document = XML.parse_html(json["content_html"].as_s)
-      nodeset = document.xpath_nodes(%q(//li[contains(@class, "feed-item-container")]))
-
-      if auto_generated
-        videos += extract_videos(nodeset)
-      else
-        videos += extract_videos(nodeset, ucid)
-      end
-    else
-      break
-    end
-  end
-
+  client = make_client(YT_URL)
   channel_html = client.get("/channel/#{ucid}/about?disable_polymer=1").body
   channel_html = XML.parse_html(channel_html)
   banner = channel_html.xpath_node(%q(//div[@id="gh-banner"]/style)).not_nil!.content
@@ -2633,6 +2604,8 @@ get "/api/v1/channels/:ucid" do |env|
               json.field "published", video.published.epoch
               json.field "publishedText", "#{recode_date(video.published)} ago"
               json.field "lengthSeconds", video.length_seconds
+              json.field "paid", video.paid
+              json.field "premium", video.premium
             end
           end
         end
@@ -2658,27 +2631,7 @@ end
       halt env, status_code: 404, response: error_message
     end
 
-    client = make_client(YT_URL)
-
-    videos = [] of SearchVideo
-    2.times do |i|
-      url = produce_channel_videos_url(ucid, page * 2 + (i - 1), auto_generated: auto_generated)
-      response = client.get(url)
-      json = JSON.parse(response.body)
-
-      if json["content_html"]? && !json["content_html"].as_s.empty?
-        document = XML.parse_html(json["content_html"].as_s)
-        nodeset = document.xpath_nodes(%q(//li[contains(@class, "feed-item-container")]))
-
-        if auto_generated
-          videos += extract_videos(nodeset)
-        else
-          videos += extract_videos(nodeset, ucid)
-        end
-      else
-        break
-      end
-    end
+    videos, count = get_60_videos(ucid, page, auto_generated)
 
     result = JSON.build do |json|
       json.array do
@@ -2708,6 +2661,8 @@ end
             json.field "published", video.published.epoch
             json.field "publishedText", "#{recode_date(video.published)} ago"
             json.field "lengthSeconds", video.length_seconds
+            json.field "paid", video.paid
+            json.field "premium", video.premium
           end
         end
       end
@@ -2755,6 +2710,8 @@ get "/api/v1/channels/search/:ucid" do |env|
             json.field "publishedText", "#{recode_date(item.published)} ago"
             json.field "lengthSeconds", item.length_seconds
             json.field "liveNow", item.live_now
+            json.field "paid", item.paid
+            json.field "premium", item.premium
           when SearchPlaylist
             json.field "type", "playlist"
             json.field "title", item.title
@@ -2876,6 +2833,8 @@ get "/api/v1/search" do |env|
             json.field "publishedText", "#{recode_date(item.published)} ago"
             json.field "lengthSeconds", item.length_seconds
             json.field "liveNow", item.live_now
+            json.field "paid", item.paid
+            json.field "premium", item.premium
           when SearchPlaylist
             json.field "type", "playlist"
             json.field "title", item.title

src/invidious/channels.cr

@@ -15,6 +15,11 @@ class ChannelVideo
     ucid:      String,
     author:    String,
   })
+
+  # TODO: Add length_seconds to channel_video
+  def length_seconds
+    return 0
+  end
 end
 
 def get_channel(id, client, db, refresh = true, pull_all_videos = true)
@@ -176,7 +181,7 @@ def produce_channel_videos_url(ucid, page = 1, auto_generated = nil)
   continuation = Base64.urlsafe_encode(continuation)
   continuation = URI.escape(continuation)
 
-  url = "/browse_ajax?continuation=#{continuation}"
+  url = "/browse_ajax?continuation=#{continuation}&gl=US&hl=en"
 
   return url
 end
@@ -184,19 +189,25 @@ end
 def get_about_info(ucid)
   client = make_client(YT_URL)
 
-  about = client.get("/user/#{ucid}/about?disable_polymer=1")
+  about = client.get("/user/#{ucid}/about?disable_polymer=1&gl=US&hl=en")
   about = XML.parse_html(about.body)
 
-  if !about.xpath_node(%q(//span[@class="qualified-channel-title-text"]/a))
-    about = client.get("/channel/#{ucid}/about?disable_polymer=1")
+  if !about.xpath_node(%q(//span[contains(@class,"qualified-channel-title-text")]/a))
+    about = client.get("/channel/#{ucid}/about?disable_polymer=1&gl=US&hl=en")
     about = XML.parse_html(about.body)
   end
 
-  if !about.xpath_node(%q(//span[@class="qualified-channel-title-text"]/a))
+  if !about.xpath_node(%q(//span[contains(@class,"qualified-channel-title-text")]/a))
     raise "User does not exist."
   end
 
-  author = about.xpath_node(%q(//span[@class="qualified-channel-title-text"]/a)).not_nil!.content
+  sub_count = about.xpath_node(%q(//span[contains(text(), "subscribers")]))
+  if sub_count
+    sub_count = sub_count.content.delete(", subscribers").to_i?
+  end
+  sub_count ||= 0
+
+  author = about.xpath_node(%q(//span[contains(@class,"qualified-channel-title-text")]/a)).not_nil!.content
   ucid = about.xpath_node(%q(//link[@rel="canonical"])).not_nil!["href"].split("/")[-1]
 
   # Auto-generated channels
@@ -207,5 +218,37 @@ def get_about_info(ucid)
     auto_generated = true
   end
 
-  return {author, ucid, auto_generated}
+  return {author, ucid, auto_generated, sub_count}
+end
+
+def get_60_videos(ucid, page, auto_generated)
+  count = 0
+  videos = [] of SearchVideo
+
+  client = make_client(YT_URL)
+
+  2.times do |i|
+    url = produce_channel_videos_url(ucid, page * 2 + (i - 1), auto_generated: auto_generated)
+    response = client.get(url)
+    json = JSON.parse(response.body)
+
+    if json["content_html"]? && !json["content_html"].as_s.empty?
+      document = XML.parse_html(json["content_html"].as_s)
+      nodeset = document.xpath_nodes(%q(//li[contains(@class, "feed-item-container")]))
+
+      if !json["load_more_widget_html"]?.try &.as_s.empty?
+        count += 30
+      end
+
+      if auto_generated
+        videos += extract_videos(nodeset)
+      else
+        videos += extract_videos(nodeset, ucid)
+      end
+    else
+      break
+    end
+  end
+
+  return videos, count
 end

src/invidious/comments.cr

@@ -109,15 +109,13 @@ def template_youtube_comments(comments)
       </div>
       <div class="pure-u-20-24 pure-u-md-22-24">
         <p>
-          <a href="javascript:void(0)" onclick="toggle_parent(this)">[ - ]</a> 
           <b>
             <a href="#{child["authorUrl"]}">#{child["author"]}</a>
           </b> 
-          <div>
           <p style="white-space:pre-wrap">#{child["contentHtml"]}</p>
           #{recode_date(Time.epoch(child["published"].as_i64))} ago
           | 
-          <i class="icon ion-ios-thumbs-up"></i> #{child["likeCount"]} 
+          <i class="icon ion-ios-thumbs-up"></i> #{number_with_separator(child["likeCount"])} 
         </p>
         #{replies_html}
       </div>
@@ -131,7 +129,7 @@ def template_youtube_comments(comments)
       <div class="pure-u-1">
         <p>
           <a href="javascript:void(0)" data-continuation="#{comments["continuation"]}"
-            onclick="get_youtube_replies(this)">Load more</a>
+            onclick="get_youtube_replies(this, true)">Load more</a>
         </p>
       </div>
     </div>
@@ -160,7 +158,7 @@ def template_reddit_comments(root)
       <p>
         <a href="javascript:void(0)" onclick="toggle_parent(this)">[ - ]</a> 
         <b><a href="https://www.reddit.com/user/#{author}">#{author}</a></b> 
-        #{score} points 
+        #{number_with_separator(score)} points 
         #{recode_date(child.created_utc)} ago
       </p>
       <div>

src/invidious/helpers/helpers.cr

@@ -2,6 +2,7 @@ class Config
   YAML.mapping({
     crawl_threads:   Int32,
     channel_threads: Int32,
+    feed_threads:    Int32,
     video_threads:   Int32,
     db:              NamedTuple(
       user: String,
@@ -357,6 +358,18 @@ def extract_items(nodeset, ucid = nil)
         live_now = false
       end
 
+      if node.xpath_node(%q(.//span[text()="Premium"]))
+        premium = true
+      else
+        premium = false
+      end
+
+      if node.xpath_node(%q(.//span[contains(text(), "Get YouTube Premium")]))
+        paid = true
+      else
+        paid = false
+      end
+
       items << SearchVideo.new(
         title,
         id,
@@ -367,7 +380,9 @@ def extract_items(nodeset, ucid = nil)
         description,
         description_html,
         length_seconds,
-        live_now
+        live_now,
+        paid,
+        premium
       )
     end
   end

src/invidious/helpers/utils.cr

@@ -40,6 +40,23 @@ def decode_length_seconds(string)
   return length_seconds
 end
 
+def recode_length_seconds(time)
+  if time <= 0
+    return ""
+  else
+    time = time.seconds
+    text = "#{time.minutes.to_s.rjust(2, '0')}:#{time.seconds.to_s.rjust(2, '0')}"
+
+    if time.hours > 0
+      text = "#{time.hours.to_s.rjust(2, '0')}:#{text}"
+    end
+
+    text = text.lchop('0')
+
+    return text
+  end
+end
+
 def decode_time(string)
   time = string.try &.to_f?
 
@@ -138,6 +155,25 @@ def number_with_separator(number)
   number.to_s.reverse.gsub(/(\d{3})(?=\d)/, "\\1,").reverse
 end
 
+def number_to_short_text(number)
+  seperated = number_with_separator(number).gsub(",", ".").split("")
+  text = seperated.first(2).join
+
+  if seperated[2]? && seperated[2] != "."
+    text += seperated[2]
+  end
+
+  text = text.rchop(".0")
+
+  if number / 1000000 != 0
+    text += "M"
+  elsif number / 1000 != 0
+    text += "K"
+  end
+
+  text
+end
+
 def arg_array(array, start = 1)
   if array.size == 0
     args = "NULL"
@@ -238,3 +274,9 @@ def write_var_int(value : Int)
 
   return bytes
 end
+
+def sha256(text)
+  digest = OpenSSL::Digest.new("SHA256")
+  digest << text
+  return digest.hexdigest
+end

src/invidious/jobs.cr

@@ -104,6 +104,44 @@ def refresh_videos(db)
   end
 end
 
+def refresh_feeds(db, max_threads = 1)
+  max_channel = Channel(Int32).new
+
+  spawn do
+    max_threads = max_channel.receive
+    active_threads = 0
+    active_channel = Channel(Bool).new
+
+    loop do
+      db.query("SELECT email FROM users") do |rs|
+        rs.each do
+          email = rs.read(String)
+          view_name = "subscriptions_#{sha256(email)[0..7]}"
+
+          if active_threads >= max_threads
+            if active_channel.receive
+              active_threads -= 1
+            end
+          end
+
+          active_threads += 1
+          spawn do
+            begin
+              db.exec("REFRESH MATERIALIZED VIEW #{view_name}")
+            rescue ex
+              STDOUT << "REFRESH " << email << " : " << ex.message << "\n"
+            end
+
+            active_channel.send(true)
+          end
+        end
+      end
+    end
+  end
+
+  max_channel.send(max_threads)
+end
+
 def pull_top_videos(config, db)
   if config.dl_api_key
     DetectLanguage.configure do |dl_config|

src/invidious/playlists.cr

@@ -65,6 +65,11 @@ def fetch_playlist_videos(plid, page, video_count, continuation = nil)
       nodeset = document.xpath_nodes(%q(.//tr[contains(@class, "pl-video")]))
 
       videos = extract_playlist(plid, nodeset, 0)
+      if continuation
+        until videos[0].id == continuation
+          videos.shift
+        end
+      end
     end
   end
 

src/invidious/search.cr

@@ -10,6 +10,8 @@ class SearchVideo
     description_html: String,
     length_seconds:   Int32,
     live_now:         Bool,
+    paid:             Bool,
+    premium:          Bool,
   })
 end
 

src/invidious/users.cr

@@ -119,6 +119,15 @@ def get_user(sid, client, headers, db, refresh = true)
 
       db.exec("INSERT INTO users VALUES (#{args}) \
       ON CONFLICT (email) DO UPDATE SET id = users.id || $1, updated = $2, subscriptions = $4", user_array)
+
+      begin
+        view_name = "subscriptions_#{sha256(user.email)[0..7]}"
+        PG_DB.exec("CREATE MATERIALIZED VIEW #{view_name} AS \
+        SELECT * FROM channel_videos WHERE \
+        ucid = ANY ((SELECT subscriptions FROM users WHERE email = '#{user.email}')::text[]) \
+        ORDER BY published DESC;")
+      rescue ex
+      end
     end
   else
     user = fetch_user(sid, client, headers, db)
@@ -129,6 +138,15 @@ def get_user(sid, client, headers, db, refresh = true)
 
     db.exec("INSERT INTO users VALUES (#{args}) \
     ON CONFLICT (email) DO UPDATE SET id = users.id || $1, updated = $2, subscriptions = $4", user_array)
+
+    begin
+      view_name = "subscriptions_#{sha256(user.email)[0..7]}"
+      PG_DB.exec("CREATE MATERIALIZED VIEW #{view_name} AS \
+      SELECT * FROM channel_videos WHERE \
+      ucid = ANY ((SELECT subscriptions FROM users WHERE email = '#{user.email}')::text[]) \
+      ORDER BY published DESC;")
+    rescue ex
+    end
   end
 
   return user

src/invidious/videos.cr

@@ -407,6 +407,23 @@ class Video
     return @player_json.not_nil!
   end
 
+  def paid
+    reason = self.player_response["playabilityStatus"]?.try &.["reason"]?
+
+    if reason == "This video requires payment to watch."
+      paid = true
+    else
+      paid = false
+    end
+
+    return paid
+  end
+
+  def premium
+    premium = self.player_response.to_s.includes? "Get YouTube without the ads."
+    return premium
+  end
+
   def captions
     captions = [] of Caption
     if player_response["captions"]?
@@ -434,6 +451,10 @@ class Video
     return description
   end
 
+  def length_seconds
+    return self.info["length_seconds"].to_i
+  end
+
   add_mapping({
     id:   String,
     info: {
@@ -456,7 +477,9 @@ class Video
     is_family_friendly: Bool,
     genre:              String,
     genre_url:          String,
-    license:            {
+    license:            String,
+    sub_count_text:     String,
+    author_thumbnail:   {
       type:    String,
       default: "",
     },
@@ -493,8 +516,8 @@ def get_video(id, db, proxies = {} of String => Array({ip: String, port: Int32})
         args = arg_array(video_array[1..-1], 2)
 
         db.exec("UPDATE videos SET (info,updated,title,views,likes,dislikes,wilson_score,\
-          published,description,language,author,ucid, allowed_regions, is_family_friendly,\
-          genre, genre_url, license)\
+          published,description,language,author,ucid,allowed_regions,is_family_friendly,\
+          genre,genre_url,license,sub_count_text,author_thumbnail)\
           = (#{args}) WHERE id = $1", video_array)
       rescue ex
         db.exec("DELETE FROM videos * WHERE id = $1", id)
@@ -571,11 +594,8 @@ def fetch_video(id, proxies)
             info = HTTP::Params.parse(client.get("/get_video_info?video_id=#{id}&ps=default&eurl=&gl=US&hl=en&disable_polymer=1").body)
             if !info["reason"]?
               bypass_channel.send(proxy)
-            else
-              bypass_channel.send(nil)
+              break
             end
-
-            break
           rescue ex
           end
         end
@@ -662,11 +682,25 @@ def fetch_video(id, proxies)
   if license
     license = license.content
   else
-    license ||= ""
+    license = ""
+  end
+
+  sub_count_text = html.xpath_node(%q(//span[contains(@class, "yt-subscriber-count")]))
+  if sub_count_text
+    sub_count_text = sub_count_text["title"]
+  else
+    sub_count_text = "0"
+  end
+
+  author_thumbnail = html.xpath_node(%(//img[@alt="#{author}"]))
+  if author_thumbnail
+    author_thumbnail = author_thumbnail["data-thumb"]
+  else
+    author_thumbnail = ""
   end
 
   video = Video.new(id, info, Time.now, title, views, likes, dislikes, wilson_score, published, description,
-    nil, author, ucid, allowed_regions, is_family_friendly, genre, genre_url, license)
+    nil, author, ucid, allowed_regions, is_family_friendly, genre, genre_url, license, sub_count_text, author_thumbnail)
 
   return video
 end

src/invidious/views/channel.ecr

@@ -13,23 +13,32 @@
     </div>
 </div>
 
-<p class="h-box">
+<div class="h-box">
 <% if user %>
     <% if subscriptions.includes? ucid %>
-    <a href="/subscription_ajax?action_remove_subscriptions=1&c=<%= ucid %>&referer=<%= env.get("current_page") %>">
-        <b>Unsubscribe from <%= author %></b>
-    </a>
+    <p>
+        <a id="subscribe" onclick="unsubscribe()" class="pure-button pure-button-primary" 
+            href="/subscription_ajax?action_remove_subscriptions=1&c=<%= ucid %>&referer=<%= env.get("current_page") %>">
+            <b>Unsubscribe | <%= number_to_short_text(sub_count) %></b>
+        </a>
+    </p>
     <% else %>
-    <a href="/subscription_ajax?action_create_subscription_to_channel=1&c=<%= ucid %>&referer=<%= env.get("current_page") %>">
-        <b>Subscribe to <%= author %></b>
-    </a>
+    <p>
+        <a id="subscribe" onclick="subscribe()" class="pure-button pure-button-primary" 
+            href="/subscription_ajax?action_create_subscription_to_channel=1&c=<%= ucid %>&referer=<%= env.get("current_page") %>">
+            <b>Subscribe | <%= number_to_short_text(sub_count) %></b>
+        </a>
+    </p>
     <% end %>
 <% else %>
-    <a href="/login?referer=<%= env.get("current_page") %>">
-        <b>Login to subscribe to <%= author %></b>
-    </a>
+    <p>
+        <a id="subscribe" class="pure-button pure-button-primary" 
+            href="/login?referer=<%= env.get("current_page") %>">
+            <b>Login to subscribe to <%= author %></b>
+        </a>
+    </p>
 <% end %>
-</p>
+</div>
 
 <p class="h-box">
     <a href="https://www.youtube.com/channel/<%= ucid %>">View channel on YouTube</a>
@@ -51,8 +60,50 @@
     </div>
     <div class="pure-u-1 pure-u-md-3-5"></div>
     <div style="text-align:right;" class="pure-u-1 pure-u-md-1-5">
-    <% if videos.size == 60 %>
+    <% if count == 60 %>
         <a href="/channel/<%= ucid %>?page=<%= page + 1 %>">Next page</a>
     <% end %>
     </div>
 </div>
+
+<script>
+document.getElementById("subscribe")["href"] = "javascript:void(0);"
+
+function subscribe() {
+    var url = "/subscription_ajax?action_create_subscription_to_channel=1&c=<%= ucid %>&referer=<%= env.get("current_page") %>";
+    var xhr = new XMLHttpRequest();
+    xhr.responseType = "json";
+    xhr.timeout = 20000;
+    xhr.open("GET", url, true);
+    xhr.send();
+
+    xhr.onreadystatechange = function() {
+        if (xhr.readyState == 4) {
+            if (xhr.status == 200) {
+                subscribe_button = document.getElementById("subscribe");
+                subscribe_button.onclick = unsubscribe;
+                subscribe_button.innerHTML = '<b>Unsubscribe | <%= number_to_short_text(sub_count) %></b>'
+            }
+        }
+    }
+}
+
+function unsubscribe() {
+    var url = "/subscription_ajax?action_remove_subscriptions=1&c=<%= ucid %>&referer=<%= env.get("current_page") %>";
+    var xhr = new XMLHttpRequest();
+    xhr.responseType = "json";
+    xhr.timeout = 20000;
+    xhr.open("GET", url, true);
+    xhr.send();
+
+    xhr.onreadystatechange = function() {
+        if (xhr.readyState == 4) {
+            if (xhr.status == 200) {
+                subscribe_button = document.getElementById("subscribe");
+                subscribe_button.onclick = subscribe;
+                subscribe_button.innerHTML = '<b>Subscribe | <%= number_to_short_text(sub_count) %></b>'
+            }
+        }
+    }
+}
+</script>

src/invidious/views/components/item.ecr

@@ -22,7 +22,10 @@
         <a style="width:100%;" href="<%= url %>">
             <% if env.get?("user") && env.get("user").as(User).preferences.thin_mode %>
             <% else %>
-            <img style="width:100%;" src="/vi/<%= item.videos[0]?.try &.id %>/mqdefault.jpg"/>
+            <div class="thumbnail">
+                <img class="thumbnail" src="/vi/<%= item.videos[0]?.try &.id %>/mqdefault.jpg"/>
+                <p class="length"><%= recode_length_seconds(item.videos[0]?.try &.length_seconds || 0) %></p>
+            </div>
             <% end %>
             <p><%= item.title %></p>
         </a>
@@ -35,23 +38,45 @@
         <a style="width:100%;" href="/watch?v=<%= item.id %>&list=<%= item.mixes[0] %>">
             <% if env.get?("user") && env.get("user").as(User).preferences.thin_mode %>
             <% else %>
-            <img style="width:100%;" src="/vi/<%= item.id %>/mqdefault.jpg"/>
+            <div class="thumbnail">
+                <img class="thumbnail" src="/vi/<%= item.id %>/mqdefault.jpg"/>
+                <p class="length"><%= recode_length_seconds(item.length_seconds) %></p>
+            </div>
             <% end %>
             <p><%= item.title %></p>
         </a>
         <p>
             <b><a style="width:100%;" href="/channel/<%= item.ucid %>"><%= item.author %></a></b>
         </p>
-    <% else %>
-        <% if item.responds_to?(:playlists) && !item.playlists.empty? %>
-        <% params = "&list=#{item.playlists[0]}" %>
-        <% else %>
-        <% params = nil %>
-        <% end %>
-        <a style="width:100%;" href="/watch?v=<%= item.id %><%= params %>">
+    <% when PlaylistVideo %>
+        <a style="width:100%;" href="/watch?v=<%= item.id %>&list=<%= item.playlists[0] %>">
             <% if env.get?("user") && env.get("user").as(User).preferences.thin_mode %>
             <% else %>
-            <img style="width:100%;" src="/vi/<%= item.id %>/mqdefault.jpg"/>
+            <div class="thumbnail">
+                <img class="thumbnail" src="/vi/<%= item.id %>/mqdefault.jpg"/>
+                <p class="length"><%= recode_length_seconds(item.length_seconds) %></p>
+            </div>
+            <% end %>
+            <p><%= item.title %></p>
+        </a>
+        <% if item.responds_to?(:live_now) && item.live_now %>
+        <p>LIVE</p>
+        <% end %>
+        <p>
+            <b><a style="width:100%;" href="/channel/<%= item.ucid %>"><%= item.author %></a></b>
+        </p>
+        
+        <% if Time.now - item.published > 1.minute %>
+        <h5>Shared <%= recode_date(item.published) %> ago</h5>
+        <% end %>
+    <% else %>
+        <a style="width:100%;" href="/watch?v=<%= item.id %>">
+            <% if env.get?("user") && env.get("user").as(User).preferences.thin_mode %>
+            <% else %>
+            <div class="thumbnail">
+                <img class="thumbnail" src="/vi/<%= item.id %>/mqdefault.jpg"/>
+                <p class="length"><%= recode_length_seconds(item.length_seconds) %></p>
+            </div>
             <% end %>
             <p><%= item.title %></p>
         </a>

src/invidious/views/watch.ecr

@@ -92,20 +92,23 @@
         <% if user %>
             <% if subscriptions.includes? video.ucid %>
             <p>
-                <a href="/subscription_ajax?action_remove_subscriptions=1&c=<%= video.ucid %>&referer=<%= env.get("current_page") %>">
-                    <b>Unsubscribe from <%= video.author %></b>
+                <a id="subscribe" onclick="unsubscribe()" class="pure-button pure-button-primary" 
+                    href="/subscription_ajax?action_remove_subscriptions=1&c=<%= video.ucid %>&referer=<%= env.get("current_page") %>">
+                    <b>Unsubscribe | <%= video.sub_count_text %></b>
                 </a>
             </p>
             <% else %>
             <p>
-                <a href="/subscription_ajax?action_create_subscription_to_channel=1&c=<%= video.ucid %>&referer=<%= env.get("current_page") %>">
-                    <b>Subscribe to <%= video.author %></b>
+                <a id="subscribe" onclick="subscribe()" class="pure-button pure-button-primary" 
+                    href="/subscription_ajax?action_create_subscription_to_channel=1&c=<%= video.ucid %>&referer=<%= env.get("current_page") %>">
+                    <b>Subscribe | <%= video.sub_count_text %></b>
                 </a>
             </p>
             <% end %>
         <% else %>
             <p>
-                <a href="/login?referer=<%= env.get("current_page") %>">
+                <a id="subscribe" class="pure-button pure-button-primary" 
+                    href="/login?referer=<%= env.get("current_page") %>">
                     <b>Login to subscribe to <%= video.author %></b>
                 </a>
             </p>
@@ -118,15 +121,12 @@
             </div>
             <hr>
             <div id="comments">
-                <h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>
             </div>
         </div>
     </div>
     <div class="pure-u-1 pure-u-md-1-5">
         <% if plid %>
         <div id="playlist" class="h-box">
-            <h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>
-            <hr>
         </div>
         <% end %>
 
@@ -137,7 +137,10 @@
             <a href="/watch?v=<%= rv["id"] %>">
                 <% if preferences && preferences.thin_mode %>
                 <% else %>
-                <img style="width:100%;" src="/vi/<%= rv["id"] %>/mqdefault.jpg">
+                <div class="thumbnail">
+                    <img class="thumbnail" src="/vi/<%= rv["id"] %>/mqdefault.jpg">
+                    <p class="length"><%= recode_length_seconds(rv["length_seconds"]?.try &.to_i? || 0) %></p>
+                </div>
                 <% end %>
                 <p style="width:100%"><%= rv["title"] %></p>
                 <p>
@@ -152,8 +155,63 @@
 </div>
 
 <script>
+function number_with_separator(val) {
+  while (/(\d+)(\d{3})/.test(val.toString())) {
+    val = val.toString().replace(/(\d+)(\d{3})/, "$1" + "," + "$2");
+  }
+  return val;
+}
+
+subscribe_button = document.getElementById("subscribe");
+if (subscribe_button.getAttribute('onclick')) {
+    subscribe_button["href"] = "javascript:void(0);";
+}
+
+function subscribe() {
+    var url = "/subscription_ajax?action_create_subscription_to_channel=1&c=<%= video.ucid %>&referer=<%= env.get("current_page") %>";
+    var xhr = new XMLHttpRequest();
+    xhr.responseType = "json";
+    xhr.timeout = 20000;
+    xhr.open("GET", url, true);
+    xhr.send();
+
+    xhr.onreadystatechange = function() {
+        if (xhr.readyState == 4) {
+            if (xhr.status == 200) {
+                subscribe_button = document.getElementById("subscribe");
+                subscribe_button.onclick = unsubscribe;
+                subscribe_button.innerHTML = '<b>Unsubscribe | <%= video.sub_count_text %></b>'
+            }
+        }
+    }
+}
+
+function unsubscribe() {
+    var url = "/subscription_ajax?action_remove_subscriptions=1&c=<%= video.ucid %>&referer=<%= env.get("current_page") %>";
+    var xhr = new XMLHttpRequest();
+    xhr.responseType = "json";
+    xhr.timeout = 20000;
+    xhr.open("GET", url, true);
+    xhr.send();
+
+    xhr.onreadystatechange = function() {
+        if (xhr.readyState == 4) {
+            if (xhr.status == 200) {
+                subscribe_button = document.getElementById("subscribe");
+                subscribe_button.onclick = subscribe;
+                subscribe_button.innerHTML = '<b>Subscribe | <%= video.sub_count_text %></b>'
+            }
+        }
+    }
+}
+
 <% if plid %>
 function get_playlist() {
+    playlist = document.getElementById("playlist");
+    playlist.innerHTML = ' \
+      <h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3> \
+      <hr>'
+
     var plid = "<%= plid %>"
 
     if (plid.startsWith("RD")) {
@@ -171,7 +229,6 @@ function get_playlist() {
     xhr.onreadystatechange = function() {
         if (xhr.readyState == 4) {
             if (xhr.status == 200) {
-                playlist = document.getElementById("playlist");
                 playlist.innerHTML = xhr.response.playlistHtml;
                 
                 if (xhr.response.nextVideo) {
@@ -185,6 +242,9 @@ function get_playlist() {
                             <% if params[:autoplay] %>
                             + "&autoplay=1"
                             <% end %>
+                            <% if params[:speed] %>
+                            + "&speed=<%= params[:speed] %>"
+                            <% end %>
                         );
                     });
                 }
@@ -208,6 +268,11 @@ get_playlist();
 <% end %>
 
 function get_reddit_comments() {
+  comments = document.getElementById("comments");
+  var fallback = comments.innerHTML;
+  comments.innerHTML =
+    '<h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>';
+
   var url = "/api/v1/comments/<%= video.id %>?source=reddit&format=html";
   var xhr = new XMLHttpRequest();
   xhr.responseType = "json";
@@ -218,7 +283,6 @@ function get_reddit_comments() {
   xhr.onreadystatechange = function() {
     if (xhr.readyState == 4) {
       if (xhr.status == 200) {
-        comments = document.getElementById("comments");
         comments.innerHTML = ' \
             <div> \
                 <h3> \
@@ -246,8 +310,7 @@ function get_reddit_comments() {
         <% if preferences && preferences.comments[1] == "youtube" %> 
         get_youtube_comments();
         <% else %>
-        comments = document.getElementById("comments");
-        comments.innerHTML = "";
+        comments.innerHTML = fallback;
         <% end %>
       }
     }
@@ -261,6 +324,11 @@ function get_reddit_comments() {
 }
 
 function get_youtube_comments() {
+  comments = document.getElementById("comments");
+  var fallback = comments.innerHTML;
+  comments.innerHTML =
+    '<h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>';
+
   var url = "/api/v1/comments/<%= video.id %>?format=html";
   var xhr = new XMLHttpRequest();
   xhr.responseType = "json";
@@ -271,7 +339,6 @@ function get_youtube_comments() {
   xhr.onreadystatechange = function() {
     if (xhr.readyState == 4) {
       if (xhr.status == 200) {
-        comments = document.getElementById("comments");
         if (xhr.response.commentCount > 0)  {
             comments.innerHTML = ' \
                 <div> \
@@ -288,7 +355,7 @@ function get_youtube_comments() {
                 <div>{contentHtml}</div> \
                 <hr>'.supplant({
             contentHtml: xhr.response.contentHtml,
-            commentCount: commaSeparateNumber(xhr.response.commentCount)
+            commentCount: number_with_separator(xhr.response.commentCount)
             });
         } else {
           comments.innerHTML = "";
@@ -297,7 +364,6 @@ function get_youtube_comments() {
         <% if preferences && preferences.comments[1] == "youtube" %> 
         get_youtube_comments();
         <% else %>
-        comments = document.getElementById("comments");
         comments.innerHTML = "";
         <% end %>
       }
@@ -307,14 +373,13 @@ function get_youtube_comments() {
   xhr.ontimeout = function() {
     console.log("Pulling comments timed out.");
 
-    comments = document.getElementById("comments");
     comments.innerHTML =
       '<h3><center class="loading"><i class="icon ion-ios-refresh"></i></center></h3>';
     get_youtube_comments();
   };
 }
 
-function get_youtube_replies(target) {
+function get_youtube_replies(target, load_more) {
   var continuation = target.getAttribute('data-continuation');
 
   var body = target.parentNode.parentNode;
@@ -333,7 +398,19 @@ function get_youtube_replies(target) {
   xhr.onreadystatechange = function() {
     if (xhr.readyState == 4) {
       if (xhr.status == 200) {
-        body.innerHTML = xhr.response.contentHtml;
+        if (load_more) {
+            body = body.parentNode.parentNode;
+            body.removeChild(body.lastElementChild);
+            body.innerHTML += xhr.response.contentHtml;
+        } else {
+            body.innerHTML = ' \
+                <p><a href="javascript:void(0)" \
+                    onclick="hide_youtube_replies(this)">Hide replies \
+                </a></p> \
+                <div>{contentHtml}</div>'.supplant({
+                contentHtml: xhr.response.contentHtml,
+            });
+        }
       } else {
         body.innerHTML = fallback;
       }