IMPORTANT: Future of the site, and what will happen going forward

EasyV said:

This is a limitation that exists even now, and I don't know if it can be changed.

[...]

Again, downvoted comments are hidden by default for non-members right now.
The same as above apply.

Like I said, those were things shii changed in Danbooru 1. That's when I was active enough to know what OYP did differently. If they can't be reimplemented now then don't sweat it.

EasyV said:

If it's for registered users then it should be configurable just fine.

Well only registered users can post comments anyway so that should be fine? Or am I missing something?

EasyV said:

Not sure about avatars, but the 404 (and other error pages) can be configured freely as long as we set up the server (current versions of danbooru use nginx) to display the appropriate page.

Lol, the avatar thing was just shii screwing with the CSS to annoy everyone. Forget I mentioned it.

EasyV said:

Unfortunately the database is the most critical component here (obviously) so if there are some wacky data it might cause problems.

I think most of it is because the site is in development mode. There should be stuff to fix inconsistent tag counts etc. in production mode, surely.

The only serious problem is the invalid tags, which are almost certainly carryover from Shimmie allowing tags that Danbooru doesn't. And because it doesn't you can't easily find and fix them... been a problem since Danbooru 1, I remember fixing lots of them by hand. They don't actually seem to be breaking Danbooru itself and I can find them in seconds with my scripts so as long as the site's up and the API's working I wouldn't worry about it.

EasyV said:

Ideally, we would need a couple of mirrors of the state the database is in when (and if) the migration happens, so that if things break we don't lose anything. However, in theory, danbooru has a "migrate" task that should "fix" or at least bring up to date the whole thing.

Clean slate, set up a new server, mothball the old one until we know everything works?

BaronMind said:

I think most of it is because the site is in development mode. There should be stuff to fix inconsistent tag counts etc. in production mode, surely.

Well, as I said in an earlier post, I might have discovered why the site can't go into production mode...
Maybe switch to it before upgrading?

BaronMind said:

Clean slate, set up a new server, mothball the old one until we know everything works?

I'm not really sure what you mean by that, actually.

I mean what I said - copy the database to a new server, keep the old one running until we know the new one works. Probably more complicated than a straight upgrade, but the OS needs upgrading anyway, so...

Due to how Danbooru works, doing that requires additional steps that would likely not work.
It's simpler and more effective to use a copy of the current database (if possible) and simply upgrade the whole thing.
The major problem is actually migrating the data, so even if we were to use a new database to check if it works, at some point we'd still have to get the old data and there lies the major hurdle.

EasyV said:

use a copy of the current database (if possible) and simply upgrade the whole thing.

That's exactly what I said. Twice. Copy the database to a new server, upgrade everything. Leave the old server running (as in, take it offline but don't delete it) as a backup until we know the new one is working. If it isn't, return to step 1.

That makes sure nothing gets accidentally deleted or overwritten until we're sure the upgrade is a success. There will be no "new database" or "old data" at any step.

Mister English that doesn't let cute Cirno get mister message across PUFFFFFFF!

So a lot of those configuration changes (ads, tag searching, etc) was more easily customizable on danbooru 1 than 2.

The ads are manually removed from the html layouts. I believe registered users can search more than 2 tags. (this is hard coded into the ruby controllers).

Basically anything you want to configure is going to rest within the ruby web frontend stuff. It's not difficult to read and to make minor tweaks in my opinion. With 0 background in ruby I was able to make some very basic changes, although I did break everything trying to incorporate captcha so there's that. >_>

BaronMind said:

That's exactly what I said. Twice.

Yunyaaaaah DX

About configurations, admittedly I didn't look too much into it yet. But since I will have to learn how it works to enable some features currently unavailable without an AWS account, at some point I should have a relatively good grasp about what can and what cannot be done.

Speaking of services, I tried to install the Archives service which is used to manage post and pool history (I find them to be useful features and in fact I use them; also OYP currently has Archive's tables in its database, so installing it is useful even just to keep OYP's history complete) and needless to say it required quite a number of changes. Like, if the current release of Danbooru (production-2019.11.06-012210-utc commit e031fb0) is used, it complains that Archive's dependencies are too old. And that's just the tip of the iceberg.

Truly a magnificent application (and it's not even the worst out there.)

This is not really on-topic, but since it's about Danbooru I thought I'd leave it here for future reference and for letting other people know in case they might be interested.
Apparently, Danbooru uses AWS only for its queue system. It makes sense to have it for a site with huge numbers of visitors, since certain actions can be queued there and then the other end of the queue can process the messages at its own pace, but for a small site (like OYP) the "enqueing speed" is the same (if not lower) than the "processing speed" of the server, so there's a good chance that it's possible to "disable" AWS just by configuring the processor to accept messages directly from Danbooru rather than from the cloud (e.g. by changing the processor from a daemon to an actual HTTP servlet.)

EasyV said:

I find them to be useful features and in fact I use them; also OYP currently has Archive's tables in its database, so installing it is useful even just to keep OYP's history complete

More than that, OYP's history has shown that being able to easily revert vandalism is a necessity. Though you can't actually see that anymore because post history didn't survive the migration to Danbooru 2... (So my joke danogoat_should_tag_properly tag is now credited to dano himself, among other things.)

EasyV said:

Apparently, Danbooru uses AWS only for its queue system.

That's good news, right?

Wish I was technical enough to actually help out, but this has all gone so far over my head all I can do at this point is cheer you on. Lots of sweet-sweets to mister EasyV for doing so much work-work! Can take it super duper easy!

BaronMind said:

That's good news, right?

Yes:

EasyV said:

so there's a good chance that it's possible to "disable" AWS just by configuring the processor to accept messages directly from Danbooru rather than from the cloud (e.g. by changing the processor from a daemon to an actual HTTP servlet.)

Needless to say, I have no idea how to write an HTTP server in ruby, but the details are unimportant.

Sorry for being so late!

@poweryoga and @Toawa, thank you for keeping OYP alive! I'm sorry to hear you two go, but I understand that the torch has to be passed eventually.

If OYP does take the big easy forever, I want to say thank you to all in this community for livening up OYP with easiness, lulz, etc.

As always, ゆっくりしていってね！

BTW, about keeping the site up, the best I can do is HTML and CSS. I have a little bit of knowledge in Javascript but am curious about Ruby.

Not everyone uses Discord, and I'm seldom on Google. Other messaging apps I use are Telegram (in which OYP has a group chat room) and Riot.im. If anyone wants an invite link to the TG group, please PM me.

>Telegram
Don't you need to give the program your phone number?

Akagi said:

>Telegram
Don't you need to give the program your phone number?

It is true that, like Discord and Google nowadays, a phone number is required. A temporary number will much likely suffice.

For those interested, I managed to "decloudify" the current Danbooru (well, not "current" as in "the latest commit available", but you get it...) and it seems to work.
However it doesn't update the database correctly. Since it happened pretty much "just now", I don't know if it's caused by me messing everything up while experimenting or something else.

Done that, all that's left is to remove various nagging messages.
After that I think it's just a matter of configuring the thing to be like current OYP.

EasyV said:

For those interested, I managed to "decloudify" the current Danbooru (well, not "current" as in "the latest commit available", but you get it...) and it seems to work.
However it doesn't update the database correctly. Since it happened pretty much "just now", I don't know if it's caused by me messing everything up while experimenting or something else.

Done that, all that's left is to remove various nagging messages.
After that I think it's just a matter of configuring the thing to be like current OYP.

I think a large portion of the remaining fanbase appreciate your efforts Easy. I know I do.

Canttakeiteasy said:

I think a large portion of the remaining fanbase appreciate your efforts Easy. I know I do.

Yes! Definitely! Thank you easy!

Also, relaying from the Discord:

Golden Lark said:

So I poked albert yi
he suggested the OYP danbooru woes be mentioned on the danbooru github
if anyone has the expertise to be of assistance, they are

Actually now that I know how to install it, Danbooru itself isn't really the problem here.
The fact that the database is not updating correctly could be anything.
If anything, I would have to ask them how to remove the uneeded messages.

Canttakeiteasy said:

I think a large portion of the remaining fanbase appreciate your efforts Easy. I know I do.

Thanks.

Edit:
Since this is the latest message I'll just edit it...
I resolved the issue with the database, but now I'm getting some other error about some object actually being nil instead of a proper value.
I wasn't really expecting things to go smoothly given the changes I had to make to the software, but I'm still fairly bummed. Can't take it easy...

Edit part 2:
It fixed itself (or I can't trigger it again)...
Well, as long as it works all's fine I guess.

Edit part 3:
We're getting there. Now it generates an error when trying to edit the wiki or creating a new forum post.
It might or might not do the same for comments, but I can't check because I forgot there is a time limit before new users can comment by default and I can't be bothered to change things up now on that front, so I'll wait until the time limit is over before testing comments.
I didn't check notes yet, but post-related features seems to work fine so far.

well, I think any new changes / updates should be mentioned on the github repo for sure.

However that's not really our problem, it's just the setting up + maintenance. Realistically Danbooru runs "ok" and doesn't need too much touching, but like any piece of software that actually needs maintenance if you do need to touch it you run the risk of breaking things unless you know your stuff. (and can fix it)

I think the comments piece should be either in the user controller or some other controller. If you want to edit the ruby files, I recommend getting notepad++ and just dumping the directory in there for searching. Unless you want to setup an actual IDE.

I just finished reinstalling it with the most recent commit.
The update was to see if I could "unbreak" things now that I know what to do and don't need to experiment. Things got fixed.
The reinstall was just to check if anything had changed on that front. Nothing did positively, only negatively (i.e. more errors.)
I reported some of the errors I met on github, let's see what happens.

The matter about comments was simply that I forgot to set the appropriate value in the local configuration. I did it this time and I could see that comments worked fine.
Notes did too, but pressing the "set as translated" button adds the "translated" tag, which has a different meaning than it does here. It should be easy(?) to change it to our "translation_available" (it should just be a find and replace... hopefully!)

Danbooru itself does run "ok", but the current incarnation has a number of extras we don't need, so before updating we pretty much are forced to edit our way through the codebase.

The biggest problem is that apparently I can't program in ruby, because the "decloudifying" program I wrote as a substitute for AWS works only half the times. ¯\_(ツ)_/¯
(Actually it just doesn't shut down cleanly, so issuing a restart never works. Still, it's a pain in the anyaru when you have to do that and forget that it doesn't work.)

I will just remind, that there's option to migrate to my booru engine that is php-based and is easy for me to support. And it currently handles over 100k pics no problem.

I understand that it's easier to upgrade to modern version of same engine. But support is always the problem. If you ever find yourself out of options, my option is still there.