Systematic playtesting instead of Reviewing?

General project talk
Post Reply
User avatar
worsas
Project Administrator
Posts: 2678
Joined: Wed Dec 31, 2014 1:23 am

Systematic playtesting instead of Reviewing?

Post by worsas »

Since reviewing has proved to create a major hurdle for our progress in the past and I already suggested to omit it for the sake of getting onwards with things, I think it would make sense to discuss alternatives for quality ensurance.

One possibility would be a systematic playtesting happening after the npcing stage.
A tester would be given a designation map of the area that is to be tested with the task of visiting every exterior- and interior-cell and doing a general check:

- Can I sleep here, even if I shouldn't be able to?
- Can I pick items or attack npcs without causing them to raise an alarm or attack me.

- Is it possible to access all places as intended?
- Is it too dark or too light in this interior cell?
- Are all load doors connected?
- Are there visible placement errors?

- Are there too many worthwhile objects found in this place?
- Are the enemies too easy or too hard? Too few or too many?
- Does the npc give reasonable dialogue responses?
- Does the trader offer reasonable goods?

- What is the general impression of it all and what could be improved?

- Do people react to me, being a vampire?
- Do people react to me, carrying a sickness?
- Do people react to me, being nude



The yellow points could be covered in a single, dedicated test by someone with accordingly prepared testing characters and wouldn't be part of a regular testing tour.

Maybe the best course of things would be:
1. A testing tour
2. Fixing all reported errors
3. A second testing tour by someone else
4. Fixing all reported errors
5. And so on until no errors are being reported anymore.

User avatar
roerich
Cruel Warlord
Posts: 2166
Joined: Sat Jan 03, 2015 3:10 pm
Location: Denmark

Post by roerich »

I'm all for this. Of course we should keep our high standards, but right now we could do with some faster progress.
"I don't know if you are kidding but I 100% support a Big Mouth Billy Bass in PC"
- Taniquetil

User avatar
worsas
Project Administrator
Posts: 2678
Joined: Wed Dec 31, 2014 1:23 am

Post by worsas »

There is one point, I forgot about that is part of each regular review:
- Is this cell using the correct assets?

This would be a unique check by a member with the according knowledge, mostly to catch dunmer- items or other objects that don't belong here. Very blatant errors in the exterior space are hopefully caught by the feedback of other members before they even survive until the testing stage.

User avatar
Luxray
Cat Herder
Posts: 544
Joined: Mon Jan 05, 2015 7:11 pm
Location: South-West England

Post by Luxray »

You could amend this to include an overview of cells in the CS by another modder, even if we don't do a full review, just to check through and catch anything (and fix anything) obvious.
<roerich> woah it's hot in here
<Lord Berandas> it must be Summer.
<Infragris> #hell is meant as a spam and off topic channel. Doing a great job already

User avatar
Scamp
Lesser Daedra
Posts: 465
Joined: Sat Jan 03, 2015 2:51 pm
Location: Kilkreath Mountains
Contact:

Post by Scamp »

Yes, reviewing is tedious, and problematic when no "qualified" reviewer is available.

Unfortunately, certain aspects of interior modding such as unique/expensive/foreign/otherwise special items related to a specific character whose profile has been discussed on the forums and which is linked to a particular claim are easily lost when reviewing is not done seperately on single claims, which (at least for me) always involved reading up on the background of the character in order not to accidentally remove "wrong" objects that were in fact placed on purpose.

Phew, that was a long sentence, wasn't it...

User avatar
worsas
Project Administrator
Posts: 2678
Joined: Wed Dec 31, 2014 1:23 am

Post by worsas »

Sometimes people leave things in their claims they are not sure about or that they cannot implement in a final shape, hoping the reviewer will deal with them later.

Alternatively, we could see, if we keep the reviewing stage, but allow any experienced modder to perform it. In my case that would probably lead to many floaters and bleeders going unnoticed though, but they would most definitely go unnoticed without any review on the other hand...

A systematic testing would still offer many advantages, regardless of reviewing still happening or not.

User avatar
worsas
Project Administrator
Posts: 2678
Joined: Wed Dec 31, 2014 1:23 am

Post by worsas »

I've thought about this topic again and seeing that we are currently skipping review for the exteriors anyway, we should, at least for the time being, do without the classical review and see if it makes sense to re-install it after the release. Right now everyone on the team is knowledgeable enough to avoid stylemixing and the kind of errors only uncovered in a classical review.

We currently have many loose ends that only wait to be connected with each other and we should totally do that: Put all of our finished exteriors and interiors into a single file and do a systematic testing run against it.

Also: We have only a handful of interiors that would need to be made to cover the currently finished parts of the druadach Highlands that connect us to scamps mesa for which the interiors have been finished aswell. There is only a relatively small amount of additional effort needed to unlock a large amount of content here. When the interiors for the northern part of the druadach highlands that are currently in the interior section, follow, we'll have even more.

Yeti
Lead Dev
Posts: 546
Joined: Mon Jan 05, 2015 11:38 pm

Post by Yeti »

Sounds like a good plan to me.

User avatar
Lord Berandas
PT Modder
Posts: 129
Joined: Sat Jan 03, 2015 6:57 pm
Location: Prague, Czech Republic
Contact:

Post by Lord Berandas »

So, my holidays started already. I am going to do some modding again, yay!
Do we have something completed, that could be actually playtested? Do we plan on releasing something soon?
Let's meet on IRC, I am present there all day long, let's get things moving again! ;-)

User avatar
worsas
Project Administrator
Posts: 2678
Joined: Wed Dec 31, 2014 1:23 am

Post by worsas »

I am currently occupied with learning new programs (substance suite and zbrush) next to work and in the coming year I will have very little time for modding. I definitely want to finish my cave until the end of the year, though. and perhaps run a data update with spineinsides new models and whatever else is contributed until then.

Right now we don't have testable content yet. The three remaining exterior claims in vorndgad forest need to be finished and all interiors (re)linked to their exteriors in karthwasten and karthgad.

User avatar
roerich
Cruel Warlord
Posts: 2166
Joined: Sat Jan 03, 2015 3:10 pm
Location: Denmark

Post by roerich »

I know I'll be busy until at least the 11th of January. I won't be ready to get things moving again before that, I'm afraid.
"I don't know if you are kidding but I 100% support a Big Mouth Billy Bass in PC"
- Taniquetil

User avatar
Scamp
Lesser Daedra
Posts: 465
Joined: Sat Jan 03, 2015 2:51 pm
Location: Kilkreath Mountains
Contact:

Post by Scamp »

Just some thoughts on exterior reviews.

Generally, I don't think we should postpone exterior reviews until NPCing is done; rather, we should ensure that they are truly ready for the next stage right after the exterior claim itself is considered finished.

Now, the issue I see is that we don't really have a single person here who would be a dedicated exterior reviewer reviewing all the things. I rather suggest that any modder, no matter if qualified for exterior-making or not, can post their thoughts based on actual playtesting. When two or three people have deemed the exterior good to go, it shall be moved to "finished". We could restrict one of these persons to be a qualified exterior maker.
None of these reviews have to be absolutely in-depth, just some general running-around and seeing how things fit together. Any minor placement errors and such will likely be reported at a later stage and can easily be fixed then.

User avatar
MinerMan60101
PT Modder
Posts: 426
Joined: Sun Jan 29, 2017 4:55 pm
Location: California

Post by MinerMan60101 »

I can totally do that during a weekend, just give me a ping on discord or something and I'll get right to it!
Interior and Exterior Developer | PT, TR
How games deal with alt-tabbing
OpenMW: That's cool i can wait for hours with literally zero problems.
Bethesda games: WF(BFHC@(HFB(8g6\ [CRASH]
MGE XE: Ha I guess you don't like distant land then [CRASH]

User avatar
Tristior
Project Administrator
Posts: 795
Joined: Sun May 07, 2017 2:12 pm

Post by Tristior »

To facilitate this, what if an experienced modder drew up a reasonably thorough reviewing checklist? That way, the process can be a bit more regulated whilst also being open to a wider range of people (still the pros, obviously).

Post Reply

Return to “SHotN General Discussion”