BBO Discussion Forums: Hand Evaluation - BBO Discussion Forums

Jump to content

  • 7 Pages +
  • « First
  • 3
  • 4
  • 5
  • 6
  • 7
  • You cannot start a new topic
  • You cannot reply to this topic

Hand Evaluation

#81 User is offline   Trinidad 

  • PipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 4,531
  • Joined: 2005-October-09
  • Location:Netherlands

Posted 2012-February-16, 09:38

Bluejak, I obviously don't know about your customers, but I can tell about my own customers:

The lesser players want to play matchpoint pairs. For them it is the only form of bridge.

The next thing on the scale is IMP teams of four. Now, we are talking about aspiring players or players with quite a bit of experience. When people are introduced to IMP teams, they will be somewhat surprised that you can play bridge with team mates. But after a while, they will get the hang of it.

People are only interested in an IMP pair game once they understand IMP teams. It doesn't make much sense to play IMP pairs with players who will not understand the difference between IMPs and MPs. For those players, you stick to MPs.

But those players who do understand the difference between IMPs and MPs might be interested in playing IMP pairs. These players have considerable experience with IMP team games. Then, it is pretty obvious that they want to use the method that mimics the team of four scoring best. They do not want a method that in a random way has some characteristics of IMPs and some of MPs, where EW pairs score better on a board than NS pairs and where making 3NT+1 sometimes scores exactly the same as 4M making and sometimes an IMP better. They want a method that is equivalent to a team of four when only two tables are in play.

For players who understand the difference between IMPs and MPs, these characteristics are important. They will chose for Cross-IMPs, as long as they are aware that Cross-IMPs exist.
For players who do not understand the difference between IMPs and MPs, you do not need to organize an IMP pair game. They would just see it as a complicated (and unfair) method to run a duplicate game.

So, for those who understand, you do it right and for those who don't understand, you just don't do it.

Rik
I want my opponents to leave my table with a smile on their face and without matchpoints on their score card - in that order.
The most exciting phrase to hear in science, the one that heralds the new discoveries, is not “Eureka!” (I found it!), but “That’s funny…” – Isaac Asimov
The only reason God did not put "Thou shalt mind thine own business" in the Ten Commandments was that He thought that it was too obvious to need stating. - Kenberg
0

#82 User is offline   bluejak 

  • PipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 4,686
  • Joined: 2007-August-23
  • Gender:Male
  • Location:Liverpool, UK
  • Interests:Bridge Laws, Cats, Railways, Transport timetables

Posted 2012-February-16, 10:32

 pran, on 2012-February-15, 14:49, said:

I was just wondering:

When you assign a weighted adjusted score at IMPs do you then use the weighted average of all appliccable total points result as one datum and assign the IMP result against this datum, or do you calculate the IMP results against each applicccable total points result and assign the weighted IMP average result?

I assume you agree that the first alternative corresponds to Butler scoring while the second corresponds to (normalized) IMP across the field, and that it is not insignificant which alternative is used?

Is the alternative to be used specified in EBU regulations?

I do not understand the question - or, to be pedantic, if I do understand it, the answer is No.

When we give a weighted adjusted score we use the score in the other room.
David Stevenson

Merseyside England UK
EBL TD
Currently at home
Visiting IBLF from time to time
<webjak666@gmail.com>
0

#83 User is offline   pran 

  • PipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 5,344
  • Joined: 2009-September-14
  • Location:Ski, Norway

Posted 2012-February-16, 11:58

 bluejak, on 2012-February-16, 10:32, said:

I do not understand the question - or, to be pedantic, if I do understand it, the answer is No.

When we give a weighted adjusted score we use the score in the other room.

What ?????

Then I have absolutely no idea what you mean with "weighted scores".

What I mean with weighted sores can best be described by this procedure: When I must assign an adjusted score and have more than one alternative results to choose between, neither of which can be excluded and each of them being more or less probable, I estimate the probability for each such alternative and then calculate the weighted average from these estimates.

My question was: Do you calculate a (single) weighted total-score average from the different alternative total-point scores and then assign the resulting IMP from this average against the total-point score obtained in the other room, or do you calculate the IMPs from each individual alternative total-point score against the total-point score obtained in the other room and then calculate a (single) resulting IMP as the weighted IMP average from these IMPs (with their respectivbe weights)?

From your answer I understand that there is no EBU regulation appliccable to determining weighted scores when results are expressed in IMPs (or VP)?
0

#84 User is offline   barmar 

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • Group: Admin
  • Posts: 21,594
  • Joined: 2004-August-21
  • Gender:Male

Posted 2012-February-16, 14:13

 Trinidad, on 2012-February-16, 09:38, said:

But those players who do understand the difference between IMPs and MPs might be interested in playing IMP pairs. These players have considerable experience with IMP team games. Then, it is pretty obvious that they want to use the method that mimics the team of four scoring best. They do not want a method that in a random way has some characteristics of IMPs and some of MPs, where EW pairs score better on a board than NS pairs and where making 3NT+1 sometimes scores exactly the same as 4M making and sometimes an IMP better. They want a method that is equivalent to a team of four when only two tables are in play.

I think that for most players, the details of the IMP scale are not really that important. The big difference between MP and IMP is the importance of overtricks/undertricks versus making/setting. When we run IMP Pair events at our club, we simply remind everyone "In this form of scoring, the important thing is making your contract, you should rarely risk your contract for overtricks, and you should stretch to bid games". This is true with both Butler and Cross-IMP scoring. The difference between them is usually just 1 or 2 IMPs for game vs. partial contracts and making vs. going down.

It's actually kind of silly advice -- most of the players who aren't aware of the difference are also not good enough to apply matchpoint strategy (they would NEVER risk the contract for an overtrick, at least not intentionally). They're usually happy whenever they make their contract, and are often surprised when we open the traveler and see that it scores poorly because they were either in the wrong contract or because they didn't go for the overtrick.

#85 User is offline   bluejak 

  • PipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 4,686
  • Joined: 2007-August-23
  • Gender:Male
  • Location:Liverpool, UK
  • Interests:Bridge Laws, Cats, Railways, Transport timetables

Posted 2012-February-16, 17:19

 pran, on 2012-February-16, 11:58, said:

What ?????

Then I have absolutely no idea what you mean with "weighted scores".

What I mean with weighted sores can best be described by this procedure: When I must assign an adjusted score and have more than one alternative results to choose between, neither of which can be excluded and each of them being more or less probable, I estimate the probability for each such alternative and then calculate the weighted average from these estimates.

My question was: Do you calculate a (single) weighted total-score average from the different alternative total-point scores and then assign the resulting IMP from this average against the total-point score obtained in the other room, or do you calculate the IMPs from each individual alternative total-point score against the total-point score obtained in the other room and then calculate a (single) resulting IMP as the weighted IMP average from these IMPs (with their respectivbe weights)?

From your answer I understand that there is no EBU regulation appliccable to determining weighted scores when results are expressed in IMPs (or VP)?

That method of assigning weighted scores has been proved to be totally wrong, and I am surprised to find anyone still doing it. Of course the EBU does not condone such a procedure. You have only to look at the effect in a MP game to realise it just does not work.

I am intrigued you say "When I must assign an adjusted score ...". Why must? Is this not common? I am worried at your whole approach to assigned adjusted scores.

Weighted scores are the norm in ruling adjustments, and the calculation into MPs or imps is done before the weighting is applied.
David Stevenson

Merseyside England UK
EBL TD
Currently at home
Visiting IBLF from time to time
<webjak666@gmail.com>
0

#86 User is offline   Trinidad 

  • PipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 4,531
  • Joined: 2005-October-09
  • Location:Netherlands

Posted 2012-February-16, 17:48

 bluejak, on 2012-February-16, 17:19, said:

Weighted scores are the norm in ruling adjustments, and the calculation into MPs or imps is done before the weighting is applied.

If I understand Sven correctly, this is exactly his point. You do -as you should- calculate the MPs or IMPs first and then you calculate the weighted average. You do not -and should not- calculate the weighted average first and then calculate the MPs or IMPs. That would be wrong. We (Sven, you and I) all agree to that.

The reason why Sven brings this up is that this wrong method is exactly what Butler scoring is based on: averaging first and then IMPing whereas Cross IMPs are equivalent to IMPing first and then weighting. If you look at it from the perspective of the other table, this wrong method is Butler. And, again from the perspective of the other table, the correct method is Cross-IMPs.

This invites the question: How do you calculate weighted adjusted scores for those customers who prefer Butler? According to the Cross IMP method (IMP or MP first and then average) or according to the Butler method (average first and then IMP or MP)?

If you -as you say- use the correct Cross IMP method, how do you explain to your customers why you use that method, rather than the Butler method that you know the customers prefer?

Rik
I want my opponents to leave my table with a smile on their face and without matchpoints on their score card - in that order.
The most exciting phrase to hear in science, the one that heralds the new discoveries, is not “Eureka!” (I found it!), but “That’s funny…” – Isaac Asimov
The only reason God did not put "Thou shalt mind thine own business" in the Ten Commandments was that He thought that it was too obvious to need stating. - Kenberg
0

#87 User is offline   Vampyr 

  • PipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 10,611
  • Joined: 2009-September-15
  • Gender:Female
  • Location:London

Posted 2012-February-16, 17:58

In my experience, teams of eight with the points added up often has the same effect as scoring two teams of four. It is very common for the four scores in question to be, eg, -450, -450, +450, -50. You have a flat board in one "match" and an eleven or however much it is loss in the other. I think that this in one reason, apart from the fact that the punters are used to it and like it, that this form of scoring is not as bad as people here are making it out to be.
I know not with what weapons World War III will be fought, but World War IV will be fought with sticks and stones -- Albert Einstein
0

#88 User is offline   PeterAlan 

  • PipPipPipPipPip
  • Group: Full Members
  • Posts: 616
  • Joined: 2010-May-03
  • Gender:Male

Posted 2012-February-17, 01:55

 Vampyr, on 2012-February-16, 17:58, said:

In my experience, teams of eight with the points added up often has the same effect as scoring two teams of four. It is very common for the four scores in question to be, eg, -450, -450, +450, -50. You have a flat board in one "match" and an eleven or however much it is loss in the other. I think that this in one reason, apart from the fact that the punters are used to it and like it, that this form of scoring is not as bad as people here are making it out to be.

It's events like this that lead people who do not understand the mathematics to believe that "this form of scoring is not as bad as people are making it out to be". Nevertheless, they are wrong, because the likelihood of such events is taken account of.
0

#89 User is offline   pran 

  • PipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 5,344
  • Joined: 2009-September-14
  • Location:Ski, Norway

Posted 2012-February-17, 02:23

 Trinidad, on 2012-February-16, 17:48, said:

If I understand Sven correctly, this is exactly his point. You do -as you should- calculate the MPs or IMPs first and then you calculate the weighted average. You do not -and should not- calculate the weighted average first and then calculate the MPs or IMPs. That would be wrong. We (Sven, you and I) all agree to that.

The reason why Sven brings this up is that this wrong method is exactly what Butler scoring is based on: averaging first and then IMPing whereas Cross IMPs are equivalent to IMPing first and then weighting. If you look at it from the perspective of the other table, this wrong method is Butler. And, again from the perspective of the other table, the correct method is Cross-IMPs.

This invites the question: How do you calculate weighted adjusted scores for those customers who prefer Butler? According to the Cross IMP method (IMP or MP first and then average) or according to the Butler method (average first and then IMP or MP)?

If you -as you say- use the correct Cross IMP method, how do you explain to your customers why you use that method, rather than the Butler method that you know the customers prefer?

Rik


You understood me perfectly and I can only wonder why David apparently did not.

You also saw the association between Butler and the incorrect way of weighting scores.

Thanks!
0

#90 User is offline   Vampyr 

  • PipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 10,611
  • Joined: 2009-September-15
  • Gender:Female
  • Location:London

Posted 2012-February-17, 02:56

 PeterAlan, on 2012-February-17, 01:55, said:

It's events like this that lead people who do not understand the mathematics to believe that "this form of scoring is not as bad as people are making it out to be". Nevertheless, they are wrong, because the likelihood of such events is taken account of.


Well then, I would have to say that I understand neither the mathematics nor what you are trying to say.
I know not with what weapons World War III will be fought, but World War IV will be fought with sticks and stones -- Albert Einstein
0

#91 User is offline   campboy 

  • PipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 2,347
  • Joined: 2009-July-21

Posted 2012-February-17, 03:59

 Vampyr, on 2012-February-16, 17:58, said:

In my experience, teams of eight with the points added up often has the same effect as scoring two teams of four. It is very common for the four scores in question to be, eg, -450, -450, +450, -50. You have a flat board in one "match" and an eleven or however much it is loss in the other. I think that this in one reason, apart from the fact that the punters are used to it and like it, that this form of scoring is not as bad as people here are making it out to be.

Sometimes these two forms of scoring give the same result on a board, true. But the right thing to consider is not the difference between how much a given board scores under the two methods, but how each method compares different boards.

Under sum-of-four-scores, -450, -450, +450, -50 is a difference of 500 or 11 IMPs away; -450, -450, -50, -50 is 1000 or 14 IMPs away. So a double swing (of this size) is only 27% more than a single swing.

Under two-teams-of-four, the first is still 11 IMPs but the second is 22 IMPs. It is not relevant that the first number is the same as before, because typically more IMPs change hands under this method, so in real terms it is much less. Now a double swing is twice as much as a single.

Under sum-of-four-times-0.7, the single swing is 350 for 8 IMPs and the double is 700 for 12 IMPs, so the double swing is worth 50% more than a single swing.
3

#92 User is offline   PeterAlan 

  • PipPipPipPipPip
  • Group: Full Members
  • Posts: 616
  • Joined: 2010-May-03
  • Gender:Male

Posted 2012-February-17, 04:40

 Vampyr, on 2012-February-17, 02:56, said:

Well then, I would have to say that I understand neither the mathematics nor what you are trying to say.

Sorry if I was being a bit cryptic. It was a bit early in the morning for me!

Taking your example as you present it, what you're implicitly saying is that 0 net at one match (pair of tables) and 500 net at the other is the same as 500 net at one match. In fact, this highlights why using the same IMP table for both is wrong, because ONE result of 500 is not the same as TWO results, one of which is 500 and the other of which is 0.

You're using your example to say "there may be 4 scores involved, but there are plenty of boards like these when it's really just 2, so we should treat it like 2", to which I'm saying "Oh no you shouldn't. Yes, there are scoring events [boards] like that, and we know all about them, but it's still correct to treat them as 4 scores and not as just 2. In particular, it's no reason to lapse back to a Teams-of-4 IMP table; we should still be using our appropriate Teams-of-8 table [whatever that may be]."

Elaborating what I'm saying, any mathematically-appropriate way of combining the 4 scores on a board and looking-up an IMP score in a Teams-of-8 table takes account of the fact that there are 4 scores involved and not just 2. Built into its mathematical construction, it implicitly "knows" perfectly well that some of these sets of 4 scores have a pair that balance each other leaving only the other 2 to contribute to the net score, and it also "knows" the proportion there are of these to sets where there are 4 different scores - it wouldn't be mathematically appropriate if it didn't.

I didn't want to go into further detail, not least because I didn't want to discuss whether pairing the scores off, as you did when you talked in terms of "matches", was appropriate. In fact, the usual Teams-of-8 scoring doesn't do this; it just adds up the total of 4 scores without worrying about how they might be paired. 420 / 170 in one match and 170 / 420 in the other is +250 & -250 for a net 0, as would be 170 / 170 at one and 420 / 420 at the other, but if you're thinking in pre-defined "match" terms one is -250 / +250 and the other 0 / 0, not necessarily the same thing.
0

#93 User is offline   PeterAlan 

  • PipPipPipPipPip
  • Group: Full Members
  • Posts: 616
  • Joined: 2010-May-03
  • Gender:Male

Posted 2012-February-17, 04:48

I'm conscious that I don't know the mathematical details of the construction of the usual IMP table. Obviously the purpose is to damp the swings that would otherwise occur under aggregate scoring, but was there a particular mathematical basis to the scale adopted, and if so can anyone point me towards it? I can speculate about what it was fitted to, but I'd prefer to know!
0

#94 User is offline   gwnn 

  • Csaba the Hutt
  • PipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 13,027
  • Joined: 2006-June-16
  • Gender:Male
  • Interests:bye

Posted 2012-February-17, 04:51

It's some sort of sigmoid function. Probably a scaled logistic scale?
... and I can prove it with my usual, flawless logic.
      George Carlin
0

#95 User is offline   Vampyr 

  • PipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 10,611
  • Joined: 2009-September-15
  • Gender:Female
  • Location:London

Posted 2012-February-17, 05:19

 campboy, on 2012-February-17, 03:59, said:

Sometimes these two forms of scoring give the same result on a board, true. But the right thing to consider is not the difference between how much a given board scores under the two methods, but how each method compares different boards.

Under sum-of-four-scores, -450, -450, +450, -50 is a difference of 500 or 11 IMPs away; -450, -450, -50, -50 is 1000 or 14 IMPs away. So a double swing (of this size) is only 27% more than a single swing.

Under two-teams-of-four, the first is still 11 IMPs but the second is 22 IMPs. It is not relevant that the first number is the same as before, because typically more IMPs change hands under this method, so in real terms it is much less. Now a double swing is twice as much as a single.

Under sum-of-four-times-0.7, the single swing is 350 for 8 IMPs and the double is 700 for 12 IMPs, so the double swing is worth 50% more than a single swing.


Right, so sometimes it is different, but does that make the method of scoring bad? I think that the effect is desirable.

Consider a common source of swings, differences in NT ranges or major opening style. I think that this is a useful example because the swings will mainly be due to luck rather than skill. Losing 11 IMPs when this happens is inconvenient, and 14 a bit more so -- but a loss of 22 IMPs may be insurmountable in a 24-board match. Just as IMPs were designed to lessen the impact of individual boards, adding the scores reduces the damage of a poor board at both tables. And of a great board at both tables. Of course you can make one pair in each direction play strong and 4 with Multi and Lucas twos, the other weak and 5 with Benji, but until that happens four tables may sometimes produce extremely large swings, and that makes the game a bit of a crap shoot.

I think really that we have ways to score pairs and ways to score teams of four, but if you are playing teams of eight or twelve or 500 you can do whatever the hell you like, as long as you come up with a score at the end.

The teams-of-eight league that I occasionally play in uses a 10-point VP scale as well. So it is all a bit different, a bit quirky maybe, but that doesn't make it inferior. And I don't understand what "mathematically appropriate" means in this context, since it is all pretty arbitrary from the start. Does a scoring method obtaining different results from another scoring method indicate that (at least) one of the methods is "inappropriate"? I don't see it.
I know not with what weapons World War III will be fought, but World War IV will be fought with sticks and stones -- Albert Einstein
0

#96 User is offline   PeterAlan 

  • PipPipPipPipPip
  • Group: Full Members
  • Posts: 616
  • Joined: 2010-May-03
  • Gender:Male

Posted 2012-February-17, 05:57

 Vampyr, on 2012-February-17, 05:19, said:

And I don't understand what "mathematically appropriate" means in this context, since it is all pretty arbitrary from the start. Does a scoring method obtaining different results from another scoring method indicate that (at least) one of the methods is "inappropriate"? I don't see it.

This just goes back to an earlier point, which was that IF you start from the position that a particular scoring method (raw score -> point conversion table - Teams-of-4 IMPs, say) is "correct" (ie has certain desirable characteristics, whatever) THEN mathematically you won't maintain correctness (those same desirable characteristics) if you use the SAME scoring method (raw score -> points conversion table) on a raw score set that's twice (three, four, four hundred times) as large.

If, however, you regard your base scoring method as totally arbitrary, then, Yes, you've got no desirable characteristics to lose. But usually the Teams-of-4 IMP scale is held to have some merits (for Teams-of-4). It can't retain those same merits in full if used unadjusted for Teams-of-8 with twice the number of scores per board.
2

#97 User is offline   campboy 

  • PipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 2,347
  • Joined: 2009-July-21

Posted 2012-February-17, 06:46

Vampyr, my previous post was not intended to compare the merits of the systems, just point out that they are not as similar as your argument would suggest.

If I were to argue the merits I would say that in my experience of teams of eight, whether you happen to get your good results on the same or different boards is a much greater source of random swing than system differences, so I prefer two-teams-of-four (or round-the-square). In inter-county matches at least, I don't think system swings double up that often, since the pairs who aren't playing the majority system are fairly spread out between the counties.

The mathematical argument was merely between my first and third examples above. Using the normal IMP scale flattens the big scores much more than it does for teams of four, simply because the big scores are bigger in teams of eight. If the amount of flattening is "right" for teams of four (and perhaps it isn't, but it was intended to be) then it makes sense to say it flattens teams of eight "too much".
4

#98 User is offline   bluejak 

  • PipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 4,686
  • Joined: 2007-August-23
  • Gender:Male
  • Location:Liverpool, UK
  • Interests:Bridge Laws, Cats, Railways, Transport timetables

Posted 2012-February-21, 09:45

 Trinidad, on 2012-February-16, 17:48, said:

If you -as you say- use the correct Cross IMP method, how do you explain to your customers why you use that method, rather than the Butler method that you know the customers prefer?

I do not understand the question. If I run a Butler pairs, it is a Butler pairs - why do I have to explain it is Cross-imps?

 pran, on 2012-February-17, 02:23, said:

You understood me perfectly and I can only wonder why David apparently did not.

You also saw the association between Butler and the incorrect way of weighting scores.

There is no association. That is just a total lack of understanding of doing things for customers.
David Stevenson

Merseyside England UK
EBL TD
Currently at home
Visiting IBLF from time to time
<webjak666@gmail.com>
0

#99 User is offline   f0rdy 

  • PipPipPip
  • Group: Full Members
  • Posts: 55
  • Joined: 2010-October-21

Posted 2012-February-21, 11:24

 bluejak, on 2012-February-21, 09:45, said:

There is no association. That is just a total lack of understanding of doing things for customers.


There is an association: they are the same method of converting from raw points to IMPs, but you deride this method in one circumstance and champion it in another.

We understand that you do this because you believe it is important to increase the enjoyment of one group of customers at the expense of the enjoyment of another group.
We understand that you believe using one method for part of the scoring of an IMP pairs game, and the other method for another part of the scoring, will keep this section of the customers happier.

What almost no-one but you (on a quick re-read of the thread) seems to believe is that the majority of "poor" players fall in the narrow band of understanding-of-scoring which leads them to be concerned about how their IMP score is calculated, concerned about what IMP scale is being used, but unconcerned about how the score for their ruling is calculated, and unable to understand a relatively simple means of scoring IMP pairs.
The belief (that this set of people isn't quite small) is at odds with the statements from experience by a couple of tournament organisers, and I think the instinct of a lot of those posting here.
0

#100 User is offline   bluejak 

  • PipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 4,686
  • Joined: 2007-August-23
  • Gender:Male
  • Location:Liverpool, UK
  • Interests:Bridge Laws, Cats, Railways, Transport timetables

Posted 2012-February-21, 13:53

When you compare apples with oranges, the results are fairly meaningless. Of course it is easy to understand weighted scores, and not to understand Cross-imps because they are totally unlike. The advantage of Butler is that players understand their score is calculated from a datum. The advantage of weighted scores is that they understand the scores are calculated from the score in the other room. But Cross-imps is confusing because there is no other score to calculate against so they cannot do it for themselves the way they can with Butler or weighted scores.

Anyone who thinks weighted scores are similar to Cross-imps in a poor player's mind has not thought it through.
David Stevenson

Merseyside England UK
EBL TD
Currently at home
Visiting IBLF from time to time
<webjak666@gmail.com>
0

  • 7 Pages +
  • « First
  • 3
  • 4
  • 5
  • 6
  • 7
  • You cannot start a new topic
  • You cannot reply to this topic

15 User(s) are reading this topic
0 members, 15 guests, 0 anonymous users