In the preface to 'A Pragmatic Legal Expert System' Popple states that
'The history of the development of legal expert systems has, for the most part, been characterized by the development and implementation of complex models of legal reasoning.'[1]
He continues
'This book aims to show that a legal expert system need not be based upon a complex model of legal reasoning in order to produce useful advice. It advocates a pragmatic approach to the design of legal expert systems ... '[2]
To those unfamiliar with the field of artificial intelligence and law, the claim that legal expert systems embody complex models of legal reasoning might seem extremely exciting - promising a wide number of tantalising possibilities. To legal theorists, legal practitioners, philosophers, cognitive scientists and psychologists amongst others, this claim would probably sound more dubious! What is striking about the models of legal reasoning embodied in legal expert systems is rather their simplicity! Hence, the claim that legal expert systems need not be based on 'a complex model of legal reasoning' to produce useful advice, that such systems can produce useful advice based on even simpler models of legal reasoning, is immediately intriguing.

Both those familiar and those unfamiliar with the field of artificial intelligence and law can find aspects of interest in Popple's 'A Pragmatic Legal Expert System'. The work is clearly written and logically structured into five chapters. The first chapter, 'Legal expert systems', discusses the field of artificial intelligence and law and major research approaches in the field. This provides that background for chapter 2, 'A pragmatic approach to case law', in which Popple argues for his approach to the construction of legal expert systems. Chapters 3 and 4 provide more detailed discussions of the implementation and operation of SHYSTER. Chapter 5 concludes the work. While the work does contain some introductory material, it is overwhelmingly concerned with discussion of the construction and operation of one legal expert system, SHYSTER, which Popple has created. As such, the work is not specifically written as an introduction to the field nor directly aimed at those unfamiliar with the field. Nevertheless, the two chapters discussing previous research in the field of artificial intelligence and law and the theoretical underpinnings of the SHYSTER system would provide interesting and controversial reading for those new to the field.

Popple sets a large goal in the book. Underlying the whole of this work and underlying the creation of SHYSTER is Popple's argument that the construction of legal expert systems should be based upon a 'pragmatic approach'. This is a point widely emphasised by Popple. Popple claims success in the operation of SHYSTER and argues that this is a demonstration of the benefit of the pragmatic approach in the construction of legal expert systems.

Unfortunately the idea of a 'pragmatic approach' is somewhat ambiguous. An expert system developer can try and be pragmatic about the way they approach the conflicting opinions in the jurisprudential literature. Additionally, system developers can be pragmatic about the kind of legal advice which they try to program a computer to produce.

Popple is certainly pragmatic about the second. One problem with legal expert systems is that the degree of legal expertise required to use them is often too great for the novice user. Popple thus designed SHYSTER for use by legal experts. The operation of the system and the advice produced by it are designed to reflect this. This recognises that the ultimate users of a system affect a system's design; how it operates and ultimately what it is designed to do.

A degree of pragmatism is also necessary in approaching jurisprudential sources. However, here Popple is more than pragmatic. Popple rejects the importance of jurisprudence for legal expert system development. [3] According to Popple
'A legal expert system need only operate at the same level of abstraction as does a lawyer, rather than at the philosophical level of a jurisprudent.'[4]
Apart from being unclear about precisely what a 'level of abstraction' actually is, Popple attempts to support this statement by asserting that
'Many lawyers perform their work with little or no jurisprudential knowledge'[5]
and that
'many lawyers have mastered the process of legal reasoning, without having been immersed in jurisprudence.'[6]
This may come as a surprise to those involved in legal education! It could be expected that reading statutes, reading precedents, reading legal text books and other legal sources, talking to other lawyers, and presenting arguments all give jurisprudential knowledge. Knowledge not only about the particular areas of law covered by those precedents, and text books but also about the kinds of reasoning that the law embodies, the kinds of arguments, principles and policies the courts adhere to and are favourable to. It is thus difficult to imagine a lawyer who has little or no jurisprudential knowledge. Many lawyers might not have much direct knowledge of those writers whose speciality includes theorising about legal reasoning itself, but this is a very different claim than that lawyers have little or no jurisprudential knowledge. Rejecting outright the value of jurisprudence, especially to the extent it discusses the processes of legal reasoning, on the ground that practising lawyers are unaware of it is analogous to rejecting the relevance of work in philosophy, cognitive science and psychology for the construction of artificially intelligent machines on the ground that billions of people successfully think and reason everyday without reference to or knowledge of that work!
 

On this basis, Popple rejects the relevance of all jurisprudence to the construction of legal expert systems and states that the pragmatic approach is
'based on the way in which lawyers deal with the law on a day-to-day basis'
Unfortunately, no evidence is provided for this. Popple claims that while
'all legal expert systems necessarily make assumptions about the nature of law and legal reasoning, it does not follow that they must conform to some jurisprudential theory.'[7]
It is difficult to make sense of this. If assumptions are being made about the law and legal reasoning then it would seem that theories are being constructed about the way law and legal reasoning operate - a form of jurisprudential theory. Perhaps this would be easier to accept if it were read to mean that legal expert systems do not have to conform to existing jurisprudential theory. However, this greatly changes the nature of the claim. For then legal theory cannot be rejected outright, but only after evaluation.

In rejecting the relevance of jurisprudence to the construction of legal expert systems and in advocating a pragmatic approach, amongst other things Popple rejects the need for 'deep conceptual models' of law. Popple states that
'Deep conceptual models of legal reasoning, like jurisprudence, operate at too high a level of abstraction to be of use in legal expert system development.'[8]
An examination of 'deep conceptual models' of law would more than likely result in scepticism of how deep such models actually are. In any case it is unfortunate that no real analysis of the benefits and trade offs is provided concerning the development of deep conceptual models. Popple borders close to rejecting these models without acknowledging the limits that this might place on the operation of a legal expert system.

Having thus established the theoretical outlook that constitutes the 'pragmatic approach' and which underpins the SHYSTER system, Popple presents a detailed discussion of the implementation and testing of the system. The results from numerous tests presented to SHYSTER and the advice produced by the system are included in the book. From these results Popple concludes that "SHYSTER has proved remarkably good at choosing a result".[9] Although he does note that the system is «quite inconsistent ... in the quality of its choice of cases.» [10]

According to the statistics cited by Popple, SHYSTER does indeed appear to have had high success in predicting the outcome of cases. [11] This is impressive. However, it must always be born in mind how far this is from what lawyers actually do. Popple says that SHYSTER
'produces its advice by examining, and arguing about, the similarities and differences between cases' [12]
and that it "attempts to model the way in which lawyers argue with cases".[13] However, the system does not argue with cases or argue about similarities and differences between cases in anything like the way that human lawyers do. It is dangerously easy to adopt anthropomorphic terms when discussing computer programs and thereby assign too much intelligence to those programs. [14] Further, in discussing the construction and testing of the system, Popple notes that cases were excluded from the system. For instance one High Court of Australia case was rejected because according to Popple
'I respectfully submit (and the legal expert agrees) that [the case]... was wrongly decided .... The case was not included ... for this reason.'[15]
However, a system that reasons in an area of law should be able to use all the cases in that area in reaching its conclusions and constructing its arguments. The system should be able to distinguish cases that the authors regard as wrongly decided. This should constitute one test of the operation of the system. Certainly, excluding such problematic cases must aid the success rates reported for SHYSTER. Popple states that certain problems and some inconsistency
'indicates a need for more attributes to distinguish between cases in that area.'[16]
This statement however, undermines the claim that SHYSTER and that the pragmatic approach can answer criticisms raised by critics of legal expert systems. [17]

Despite reservations concerning claims made about the role of jurisprudence in the development of legal expert systems and despite scepticism about exactly how pragmatic the pragmatic approach to the development of legal expert systems really is, 'A Pragmatic Legal Expert System' describes an interesting research project in artificial intelligence and law. For those already interested in the field, this work will provide a valuable account of an interesting project. For those new to the field the work, and particularly the first two chapters, provides an interesting perspective on the field
.

Notes

[2] Ibid.

[3] Ibid 7.

[4] Ibid 7.

[5] Ibid 6.

[6] Ibid 7.

[7] Ibid 6.

[8] Ibid 47. Popple also states that
Deep conceptual models of legal reasoning are inappropriate for legal systems. They contribute to the difficulty of knowledge acquisition. Ibid 50With respect, this is not an argument against the benefits of deep conceptual models for legal expert systems. If systems had been constructed that provide complex legal advice to the level of a human legal expert, then that would be evidence against the need for deep conceptual models. Unfortunately that is not the case. Similarly, Popple states thatSemantic networks are of dubious use for representing case law. At the very least, their use is not consistent with the pragmatic approach to case law, which I argue, is appropriate for expert system design. Ibid 44However, while knowledge representation using semantic networks may be laborious, this is not an argument against their usefulness. Nor indeed is any claimed inconsistency with the pragmatic approach (although it is not even immediately clear why the use of semantic networks is not consistent with the semantic approach). Popple states that the pragmatic approach «makes simpler the process of knowledge acquisition», ibid xiii. Unfortunately exactly how this happens is not made clear. While the knowledge representation scheme underlying SHYSTER may be simpler than some other projects in artificial intelligence and law, as Popple acknowledges in his discussion of HYPO, this simplification has resulted in reduced reasoning abilities in SHYSTER, ibid 113-5. Unfortunately, Popple does not discuss this trade off between the complexity of the knowledge representation scheme used and the reasoning abilities of the system based on that scheme nor try to quantify ho much easier the simplification in the knowledge representation scheme underlying SHYSTER actually makes the process of knowledge representation.

[9] Ibid 246.

[10] Ibid 247.

[11] Ibid 230.

[12] Ibid xiii.

[13] Ibid xiv.

[14] Hofstadter Douglas and The Fluid Analogies Research Group, Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought (1995) Basic Books.

[15] Popple, above n 1, 220.

[16] Ibid 229.

[17] Popple argues that SHYSTER operates in an area which Mendelson (Mendelson Simon, 'An Attempted Dimensional Analysis of the Law Governing Government Appeals in Criminal Cases' p.128, in The Second International Conference on Artificial Intelligence and Law: Proceedings of the Conference (1989) ACM Press) has characterised as difficult, Popple, above n 1, 246. Popple continues and states that
Extra attributes were not added, however; the seven attributes in the Natural area are based on the seven factors identified by McMillan as being important.»It is obviously a jurisprudential choice to rely only on the factors that judges have themselves identified as important in case law. This again undermines the claim that jurisprudence is irrelevant in the pragmatic approach to constructing legal expert systems.