Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 6:40 am

Let me know if you spot anything that doesn't work like it did at the original site.

The known deficiencies at this time are:

1. [ETA: Done!] Of the old posts which have file attachments (usually images), about one in eight will show no "thumbnail" (there will be an empty frame), until I finish uploading all file thumbnails (there were 14,382 of them, totaling 579 megabytes). I paused uploading until the new host server updates its usage statistics so that I can see how badly uploading impacts the metrics (they should update Monday sometime).

2. [ETA: Done!] Likewise and for the same reason, there are no actual file attachments on old posts (i.e. if you click on an image attachment you won't be able to see the full-sized image yet, and if there's an attached Excel file or whatnot you won't be able to download it). Again, I'm waiting to see if I'm pushing the upload limits for the month or not. There are 18,967 file attachments totaling 2.66 gigabytes of data, zero of them are up on the server yet, I may have to wait until the first of next month (in a week or so) to upload them if I'm running close to this month's quota already.

[ETA: Fixed!] Try to limit searches if you can -- searching for posts by a specific poster or by date (i.e. by the "view active topics" link) should be okay, but doing an actual text search (e.g. any post containing a certain word or phrase) puts a big load on the server, and it's currently running on the smallest virtual machine size, so try to keep that to a minimum for now.

Re: Welcome back!

Mon Sep 22, 2014 2:59 pm

Since the software appears to have remembered my formatting preferences, I feel right at home.

I'll holler if I find anything odd that you haven't already mentioned.

Thanks for doing this! =D>

Re: Welcome back!

Mon Sep 22, 2014 3:07 pm

Update: All thumbnails are now in place.

It turns out that AWS doesn't charge anything for inbound web traffic (no matter the volume), they only charge for outbound (with the first 15GB of the month being free).

So there's no problem slamming the whole shebang up to the server as fast as I can upload it. I'll start uploading the 2.66GB of file attachments shortly.

Re: Welcome back!

Mon Sep 22, 2014 5:11 pm

Terry wrote:Thanks for doing this! =D>

Yes, thanks! :)

Re: Welcome back!

Mon Sep 22, 2014 6:36 pm

Woot! Goats, cookies and beer!

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 7:53 pm

Refresh my memory, but this seems a good point to review image X/Y size limits for links that are smaller than image size limits (if any?) for in-line pictures. This encourages saving large images in-line, thus eating up cost of outbound bandwidth. I think we want to have the in-line capability for non-hosted images. But let's at least not encourage them if an externally hosted image exists.

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 8:07 pm

Thank you for doing this! I was so missing DC!

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 8:19 pm

Oh, but anyway, Toto, we're home. Home! And this is my room, and you're all here. And I'm not gonna leave here ever, ever again, because I love you all, and - oh, Auntie Em - there's no place like home!

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 8:20 pm

Great. Now, methinks, it is the time to discuss having a 'thon. Say, in October, in balrog's direction. How much are the bandwidth charges?

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 8:31 pm

jlogajan wrote:Refresh my memory, but this seems a good point to review image X/Y size limits for links that are smaller than image size limits (if any?) for in-line pictures. This encourages saving large images in-line, thus eating up cost of outbound bandwidth. I think we want to have the in-line capability for non-hosted images. But let's at least not encourage them if an externally hosted image exists.

The current max for external images displayed inline in a post is 500 x 500. I think that's the phpBB default.

The problem with anything much larger is that wide images will blow the thread formatting (since it pushes the right margin out to fit the image), which especially can be a problem on mobile devices with small screens.

I'll set it to 600 x 1024 and we'll see how it goes. 600 pixels wide will still (barely) fit on an old iPhone screen (held vertically), while the image height is not so much an issue (and increasing it will allow taller images to be linked).

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 8:33 pm

Ichneumon wrote:The problem with anything much larger is that wide images will blow the thread formatting...

I can't argue for or against size limits on that account. However it confused me why there would be one limit for external images and a larger limit for in-line images.

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 8:37 pm

Ichneumon wrote:The current max for external images displayed inline in a post is 500 x 500. I think that's the phpBB default.

The problem with anything much larger is that wide images will blow the thread formatting (since it pushes the right margin out to fit the image), which especially can be a problem on mobile devices with small screens.

I'll set it to 600 x 1024 and we'll see how it goes. 600 pixels wide will still (barely) fit on an old iPhone screen (held vertically), while the image height is not so much an issue (and increasing it will allow taller images to be linked).

Ichneumon: how about setting a 3hrs time window for edits? It would not be excessively generous, but still it could provide a better chance of correcting one's illiteracies of all kinds.

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 8:42 pm

jlogajan wrote:
Ichneumon wrote:The problem with anything much larger is that wide images will blow the thread formatting...

I can't argue for or against size limits on that account. However it confused me why there would be one limit for external images and a larger limit for in-line images.

Technically both external images (those included via the [IMG] tag pointing to an image elsewhere on the internet) and attached images (those uploaded to DC via the "Upload attachment" tab) are shown "inline" (i.e. are visible in the post itself).

The difference is that when displaying an external image, the forum is at the mercy of whatever the size actually is, whereas when an image is uploaded, the forum automatically creates a 400-pixel-wide "thumbnail" version which it saves alongside the full-sized image. The thumbnail is shown in the post when you look at a thread, the full-sized image is seen if you click on the thumbnail.

Thus the limit on what sizes of images it will let you externally link to.

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 8:48 pm

GSlob wrote:Ichneumon: how about setting a 3hrs time window for edits? It would not be excessively generous, but still it could provide a better chance of correcting one's illiteracies of all kinds.

I have now set it to 3 hours (the prior limit was 1 hour).

The time limit to delete a post you've made is still 1 hour.

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 9:08 pm

Test:

Ooh, damn I'm good... :mrgreen:

That's a 1546 x 1161 externally linked image. :poms: Click on it to see the full-sized version over at the original site (someone's blog I grabbed at random via Google).

I added coding for a new image tag, "[bigimg]". Use it just like the original (and still existing) [img] tag, whenever you're attempting to link an external image and get the "too large" whine from the forum.

So just do: [bigimg]URL-of-the-external-image[/bigimg]

When this is translated to your web browser's HTML to display the page, it adds a "width=500" clause, which makes your browser fetch the image from its original location (as usual) but then displays it onscreen resized to be 500 wide (done on the fly in your computer while rendering the page).

This is why you don't want to use [bigimg] for *all* images (just the too-big ones), lest you end up with something like this when you link to small image (it resizes it UP to 500 wide -- bigimg displays *everything* 500 wide):

Here it is using the original [img] tag:
Image

Be aware that there's still a compelling reason to upload image attachments: They'll always be available as long as the DC forum itself is, whereas external images will be non-displayable if the external site goes down, goes away, changes the image URL, removes the image, etc.

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 9:10 pm

Caramelgal wrote:Oh, but anyway, Toto, we're home. Home! And this is my room, and you're all here. And I'm not gonna leave here ever, ever again, because I love you all, and - oh, Auntie Em - there's no place like home!

:lol: :grouphug:

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 9:41 pm

Interacting with the forum software after a 2 month hiatus is a stunning reminder of just how bad Yahoo Groups' interface really is.

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 10:39 pm

GSlob wrote:Great. Now, methinks, it is the time to discuss having a 'thon. Say, in October, in balrog's direction. How much are the bandwidth charges?

If we stay with AWS (where it's currently running), the charges will be based on usage. I'm not sure what typical usage will be, after most folks come back and we have a full month of metrics (beyond this initial start-up month, which ate a lot of bandwidth/IO getting the site set up), I'll be able to more accurately estimate the likely month-to-month fees.

AWS has a "free tier" for your first year to try things out, and I signed up last week, so for the next 50-ish weeks we'll be getting a break as long as we stay under certain usage thresholds.

The AWS usage fees are:

t1.micro instance (virtual machine) fees: Totally free for first year, $0.013 per Hour thereafter. So a 31-day month would have $9.67 in CPU fees. (Another possibly outdated page implies $0.02/hour, if so monthly would be $14.88). That doubles if we have to move up a tier for adequate performance, quadruples if we have to move up two tiers. Moving up a tier is like renting a larger computer to run on.

CPU load: CPU activity is free, unless the virtual machine is found to be running at 90%+ a lot of the time, in which case they make you move up to a higher tier. That shouldn't be a problem for us. At worst we'll lag at peak times but it won't be nonstop. Moving to a higher tier would be a matter of choice if we don't like things being sluggish at peak times.

EBS (basically the disk drive) fees: 30GB free for the first year (no way we'll hit that, we're at about 6GB now including operating system and all that), $0.05/GB/month storage over/beyond that. So if we have about 10GB on average, that'll be roughly $0.50 in disk fees per month after the first year.

EBS IO (basically disk activity) fees: 2 million free per month for the first year, $0.05/million over/beyond. Setting up DC I've burned through 705,893 IOs, but that's a *lot* of disk chunking (uploading many gigabytes, unpacking compressed files, etc.) which I don't expect to have to do again, and everyday use shouldn't pound the disk very hard (with the exception of searching all posts for a given word/phrase, which potentially can read 300MB at a thwack (the size of the post database table), that would be about 18,000 IOs I think (a disk IO is a 16kb chunk of data read or written, or any smaller part thereof in a single read/write request). I'm not sure how much regular usage will consume, but even if it's 20 million (enough for 1000 worst-case forum searchs) that's only $1.00 a month for this.

Inbound web traffic: Free! Forever! (Unless AWS reprices its services)

Outbound web traffic: This is one of the biggest wildcards, I have no idea what typical traffic might be like once everyone comes back and starts checking in often. Keep in mind that viewing threads with inlined, attached images will feed the image thumbnail (average thumbnail size is 40k) each time the thread is viewed (although refreshing might just show your locally cached version again). Outbound web fees are: 15GB free for the first year (unclear whether that's per month or total), $0.12/GB over/beyond that. Setting the site up, I've used 0.137GB outbound web traffic so far, but most of my activity has been inbound, not outbound. I also don't know how much data the searchbots (Google et al) suck up when they visit. I've got some logs from the old site, I'll start poking around and see if they have this kind of traffic info.

So wild guestimate of costs is something like $0.00 for the first year (or rather minimal cost for monthly overruns beyond the free allotments), then $9.67+$0.50+$1.00= $11.17/month in known fees beyond that, plus an unknown monthly fee for outbound web traffic (wild-assed generous guess of 100GB/month would be an extra $12 in monthly fees, bringing total to $24.17).

I think Central Archivist also pays an annual fee for the domain name (DarwinCentral.org), we'll apply the domain name to this server once we're sure it's where we want to live for a while. I think domain names are what, like $40/year?

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 10:47 pm

I just tried "View your own posts" and it is very slow...... took about 10 seconds to respond. Much slower than the old DC™, just so you know.

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 10:51 pm

Elmo Zoneball wrote:I just tried "View your own posts" and it is very slow...... took about 10 seconds to respond. Much slower than the old DC™, just so you know.

I've noticed that too, although "just now" I was uploading a 560MB file and unzipping another one that had been previously uploaded, so...

This "machine" has less horsepower than the original one. If need be we can easily upgrade to a beefier one, but a) we'd lose our free year if we did, and b) it'd cost about another $10 a month (which might be worth doing once we use up our freebie period).

It's possible i could also find ways to tune up how searches are done, there are numerous options (both in the forum itself, and at the system level).

Re: Welcome back! [September 2014 reboot]

Mon Sep 22, 2014 11:10 pm

All pre-existing image thumbnails and file attachments from the old site have now been uploaded.

Re: Welcome back! [September 2014 reboot]

Tue Sep 23, 2014 3:20 am

Ichneumon wrote:I added coding for a new image tag, "[bigimg]". Use it just like the original (and still existing) [img] tag, whenever you're attempting to link an external image and get the "too large" whine from the forum.

Outstanding!

Re: Welcome back! [September 2014 reboot]

Tue Sep 23, 2014 6:28 am

Mobile device auto-detect is now working (i.e. viewing the forum from your phone will now automatically use the mobile-specific skin/theme).

Re: Welcome back! [September 2014 reboot]

Tue Sep 23, 2014 7:27 am

Can you do anything with automatic login? I have to login again every time I check back unless I leave a tab open on DC.

Re: Welcome back! [September 2014 reboot]

Tue Sep 23, 2014 7:31 am

saganite wrote:Can you do anything with automatic login? I have to login again every time I check back unless I leave a tab open on DC.

Yeah, that's annoying me too, it seems to lose the session at the drop of a hat. I'll research it.