The Legal Text Search

The Le= gal Text Search

Why would searching legal texts be = any different from searching any other text?

The elementary purpose of text search is to find relevant documents bas= ed on the user's search criteria – usually a search string, comprised of= one or more words that the user considers sufficiently descriptive of the matte= r.

We'll be taking a look at something far more sophisticated, but we need= to start with the basics. So, please, bear with me.

The majority of people think about this when text search methods are mentioned:

&nbs= p; &= nbsp; &nbs= p; 1. &nb= sp; Basic text search:

= &nb= sp; = - &nbs= p; interest – matches words such as »interest«= ;, but may fail[1] to m= atch »interesting«, »interests«

More experienced users also find use of pattern-matching, where one or = more characters of the search string are replaced by one or more wildcard characters, such as the asterisk (*) and the qu= estion mark (?). The wildcard character= s are generally used when searching expressions where the keyword is used as a pa= rt of another expression, but the obvious limitation is purely mechanical R= 11; the actual meaning of the word is completely ignored.

&nbs= p; &= nbsp; &nbs= p; 2. &nb= sp; Wildcard text search:

= &nb= sp; = - &nbs= p; interest* – matches: »interest«, »int= eresting«, »interests« (since the »*« character stands for »any character«).

On the other hand, the following search string:=

= &nb= sp; = - &nbs= p; Interest? – only matches: »interest«, &raqu= o;interests« (since the »?« character stands for »any single character«).

The above methods of text search are very basic. As mentioned, the actual meaning of the words is not relevant to the search engine (due to the fact that modern search engin= es have no concept of meaning), altho= ugh it may very well be extremely relevant to the user. In the examples above = documents containing text such as »of no interest«, »highly interesting«, and »interest rate« will be matched, althou= gh they may not correspond to the meaning the user has in mind. As a result the mat= ched documents may not be what the user actually wanted to find.

Most modern search engines also employ some basic logical operations wi= th the »and«, »or« and »and not« operators, enabling the user a bit more control over the search results:

&nbs= p; &= nbsp; &nbs= p; 3. &nb= sp; The AND operator:

= &nb= sp; = - &nbs= p; world AND record – matches all docu= ments where both words are present;

&nbs= p; &= nbsp; &nbs= p; 4. &nb= sp; The OR operator:

= &nb= sp; = - &nbs= p; world OR record – matches all docu= ments where either one of the words is present;

&nbs= p; &= nbsp; &nbs= p; 5. &nb= sp; The AND NOT operator:

= &nb= sp; = - &nbs= p; world AND NOT record – matches all docu= ments where the first word is present, but not the second.

By employing both the wildcard characters and the logical operators more sophisticated search strings can be built:

= &nb= sp; = - &nbs= p; (world AND record*) AND NOT football – mat= ches all documents with the word »world« and all forms of the word »record« in them, but not those that contain the word »football«;

= &nb= sp; = - &nbs= p; pontiac AND (firebird OR gto) AND NOT used – matches all documents with the word »= pontiac« and= either the word »firebird« or »gto«, but not the word »used«.

With the logical functions text search does become more complex, with m= ore to consider when building efficient search strings, but this is usually rewarded by more accurate search results. Of course, the actual meaning of the words in the search strings is still not taken into account, even though very basic semantic rules could be writ= ten using these basic logical operators:

= &nb= sp; = - &nbs= p; In the first= case the user is interested in world records, but wishes to exclude documents related to football from the results. Is it because the user is not interes= ted in football or is it because of the overwhelming quantity of results relate= d to football? Who knows…;

= &nb= sp; = - &nbs= p; In the secon= d case the user is interested in two = Pontiac car models, but wishes to exclude documents related to the sales of used ca= rs. Again, it's difficult to guess what the user actually wants from a single search string.

There are other options already available that serve as an extension to= the basic functions. One of them is the »near« operator; it's based= on the assumption that words with a special meaning exist in close proximity:<= o:p>

&nbs= p; &= nbsp; &nbs= p; 6. &nb= sp; The NEAR operator:

= &nb= sp; = - &nbs= p; prison NEAR sentence – only those docum= ents are matched where the two words are in close proximity, while the documents where the two words are far apart are disregarded.

The threshold used to compute proximity depends on the search engine us= ed, and on the language of the document. Some search engines allow the user to specify proximity:

= &nb= sp; = - &nbs= p; prison NEAR(4) sentence – only those docum= ents are matched where the two words are not more than four words apart.

The use of proximity brings searching for strings closer to the actual meaning of searched expressions, closer to what the user actually wants to find, but sadly not even close to = any meaning. In fact, the user might think that the true meaning of the wor= ds is regarded by the search engine, when in fact this is very much not so. Th= is way the user might be relying on nonexistent facts, and might end up being = even more dissatisfied with the results.

The methods and functions of text search mentioned above have been implemented with great success in a standard known as Regular Expressions. All functions mentioned above and many additional functions have been implemented into the standard, making it a v= ery powerful text search tool. However, at the same time building a sophisticat= ed regular expression might just as well be too difficult for the average user= .

&nbs= p; &= nbsp; &nbs= p; 7. &nb= sp; The Regular Expression:

= &nb= sp; = - &nbs= p; (prison\ssentence(s){0,1})|(imprison(ed|ment(s){0,1})) – mat= ches documents where the expression »prison sentence«, »prison sentences«, »imprisoned« or »imprisonment« are used.

Regular Expressions may help in allowing the user to specify more search criteria with less typing, but with more efforts invested into learning the standard and how to use it properly. In fact, the complexity of regular expressions compared to more basic text search methods may increase exponentially with the complexity of the actual search criteria.=

Of course, even the most = powerful regular expressions do not by themselves support meaning.=

Content Providers vs. Search Providers

In general cases the meaning of words and expressions used in documents= compared to the meaning of words and expressions used in search criteria is versatile – the possibilities are practically endless. The mainstream of today's search providers (e.g. Google, Yahoo, AltaVista, MSN Search etc.) tends not to regard any meaning directly, since their main goals are not parallel= to the goals of the user.

The focus of search providers is, as follows:

= &nb= sp; = I. &nb= sp; To provide results quickly (i.e. instantly) –= any delays in the service may have a seriously negative effect on the user, so = only basic validation of the search string is made, followed by immediate execut= ion of the search. The search string is parsed – purely mechanically R= 11; simply in order to discard any syntax flaws the user might have made;

= &nb= sp; II. &n= bsp; To provide relevant results – the goal is to p= rovide as many results as possible that match the search string as closely as poss= ible, usually in the order of relevance. The methods of computing relevance are diverse, but they tend to follow some basic principles: e.g. what part (element) of the document corresponds to the search criteria, how many matc= hes there are in a document, how recently has the document and/or the site been updated, etc.;

= &nb= sp; III. &= nbsp; To provide at least a few results (in contras= t to no results at all) – search providers tend to either blame the user[2]= for the lack of results or they blame the search criteria. The user is rele= ntlessly instructed to check the search string for typing and spelling errors etc., = even if the spelling in the search criteria is correct. The search provider is confident in its abilities to find a match for any correct[3]= (and not »too exotic«) search criteria – the error messag= e, however, always suggests a flaw in the search string – ergo indirectly implicating the user[4]= .

Of course an even more important goal of the search provider is revenue – »every click counts«. Be it a link in the result list, a sponsored link or an advertisement – users are encouraged to search and click.

As far as the meaning (or the approximate meaning) of the search criter= ia is concerned, the search providers have made some progress in this area of content management. The probable (= not the actual!) meaning is derived – not from search strings directl= y, but from the relationships that exist between the search criteria, the search results, the documents that we= re actually viewed and the number of users= that have followed equal or comparable search and use patterns. E.g. after a certain number of users have used the same search criteria the ranking of results is adapted (by the search provider – either automatically or through intervention) to display more popular documents for the given set of criteria further up the list.

As mentioned, the actual meani= ng of the search criteria remains unknown. However, an »educated« guess is used to somewhat refine the search results in order to favour those that – according to the search statistics – probably match the criteria more closely.

Especially for the general (i.e. endlessly versatile) types of content = this approach is favoured by the search providers for one or more of the followi= ng reasons:

= &nb= sp; = - &nbs= p; It's cost efficient – users create the statistics themselves, simply by using the search service, the provider just needs to monitor use;

= &nb= sp; = - &nbs= p; If an approp= riate probability threshold is used for a given set of criteria, then the match between the search resu= lt(s) and the search criteria really is = as close as the average user would expect. This, of course, is no guarante= e of true meaning, but it's a more or less valid approximation;

= &nb= sp; = - &nbs= p; The statistics can be used to »manu= facture« meaning – it would generally be easier to find a sponsor for a »popular search« than it would be to find one for a less popula= r or »unpopular search« – ergo a possible (though not really serious) answer to the question, why do result lists begin with sponsored l= inks;

= &nb= sp; = - &nbs= p; The statistics can be used to create more revenue – »popular searches« are not offered to a sin= gle sponsor, but are subject to bidding. The sponsored link more likely than not belongs to a sponsor who can afford it, rather than to a sponsor that more closely matches the actual needs and expectations of the users.<= /span>

Nonetheless, the = search providers still tend to provide matching content, rather than »matchi= ng sponsors«. The competition among search providers is stiff and us= ers can be unforgiving, as some revenue-hungry publishers of printed media have experienced in the past (readers p= refer articles, not advertisements!).

Once again it's quite obvious that the above motives point to one thing – revenue. Although the = search service itself is free of charge from the user's point of view, the main go= al of the search provider remains economical. If not directly (e.g. by being financed per click), the economic aspect of the search provider is addresse= d indirectly (e.g. by building its public image, marketing other services and products e= tc.).

We've taken a look at the methods employed by some of today's search providers – their ap= proach is universal rather than focused on certain groups of content, and certainl= y is not focused on specific groups of users. The goal is to provide a free[5]= service with as little investments into in-depth research of content as possible.

The position of content provid= ers seems quite opposite – most of today's content providers provide specialised content or several collections of specialised content to their subscribers (and potential subscribers prior to that). The subscribers basically represent one or more focused groups of users. These generally ha= ve rather specific needs and expectations regarding content. The approach of regular subscribers (in contrast to non-subscribers) tends to be more syste= matic and explicit rather than intuitional or less explicit. They have subscribed= to a service on a basis of a rational decision, especially if their work depen= ds on it and if they are charged for using it.

The accuracy of search results is the key feature of content management systems in contrast to search engines. Then again – the more speciali= sed that content providers are (in respect to types of content and/or target users), the more focused they can be on content preparation, and a higher l= evel of accuracy in search results can be achieved. The limitations, though, rem= ain very obvious:

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; Specialised content – content usually covering one or only a few subjects (e.g. a specific science, such as mathematics, physics, law or medicine; or even general encyclopaedias or vocabularies). The entire subject mass in the case of a specialised content provider is highly deterministic – both concerning content preparatio= n as well as content use;

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; Specialised target users are also considered a key feature of specialised content management services. Their needs and expectations are highly deterministic, and combined with the highly deterministic characteristics of content provide a highly manageable founda= tion of content management.

We've mentioned a few areas of content management in the previous paragraph, now let's focus on one of them.

Legal content

Although law is an ancient science and has over the centuries (and millennia) gone through many dramatic changes, it remains a fairly stable accomplishment of the human civilization.

As discussed in another article[6]= one essential quality of legal documents is the characteristic known as the norm pattern. One or more of = the four elements of the norm pattern can be observed in any legal document = 211; the disposition, the sanction, the association between the= norm and the facts, and the effect of the consequences= . Most of the elements can easily be observed by any person, while some of the elements may require the observer have a more in-depth knowledge of legal content.

The fact of the matter stands solid – elements of the norm pattern exist in any unit of legal content. In fact, the existence of one or more of the elements effectively constitutes legal content. The norm pattern is also an important common characteristic of legal documents.

Ambiguity of meaning is critical= in respect to the use of legal content. Not only does it have to be avoided whenever and wherever possible, it also needs to be identified and its existence emphasised – the users of legal content should be fully awa= re of any and all possible ambiguities wherever they occur.<= /p>

In more that two millennia of law science numerous great developments a= nd discoveries have been made, yet many issues still aren't fully resolved = 211; there are several disputes between theorists on many diverse issues. This constitutes another critical characteristic of legal content that needs to be considered as the content = is used.

There are several legal (and judicial) systems in the world; the cultur= es are diverse and so are the languages. Individual legal and judicial systems have their own special characteristics, which need to be fully apparent whenever the different cultures meet (when people of different cultures interact). These diversities usually manifest themselves as legal discrepancies that need to = be addressed in some way.

Ever since the end of World War Two the progress of international law h= as been immense as the countries of the world formed various international and even super-national organizations. Legal and judicial systems were compared= to each other with great scrutiny as harmonisations and even unifications were= considered, and eventually reached. In other words – new patterns related to legal content management were discovere= d and developed.

If we sum up the characteristics of legal content we get a clearer pict= ure of the true challenges of legal content management:

= &nb= sp; 1. &nb= sp; Identifying relevant patterns – bes= ides the ever present norm pattern there are other patterns related to the contents = of legal documents, their structure and use. The majority of these still remai= n to be identified and properly researched. An in-depth knowledge of legal interpretation as well as expertise in nomotechnics is m= uch needed;

&nbs= p; &= nbsp; &nbs= p; 2. &nb= sp; Using patterns to recognise content – the recognition of patterns in legal documents should be part of content preparation, giving the content new and additional value, and let us not fo= rget – meaning. The patterns = could be of great benefit to both content preparation and content use;=

&nbs= p; &= nbsp; &nbs= p; 3. &nb= sp; Detecting discrepancies – an intercultural comparative analysis of legal documents should also be an integral part of legal content management. While the primary goal of detecting discrepancies could be in making legal harmonisation and unification more efficient, anot= her goal could be in assisting the users of legal documents when dealing with differences of legal and judicial systems of different countries of the wor= ld;

&nbs= p; &= nbsp; &nbs= p; 4. &nb= sp; Identifying theoretic disputes – a k= ey element of scientific progress is the ability of people involved to exchange ideas and to develop solutions to common problems. As long as theoretical disputes exist we should at least be aware of them, and in addition have the opportunity to study them, if not to also take part in them;

&nbs= p; &= nbsp; &nbs= p; 5. &nb= sp; Resolving ambiguity – one of the princ= ipal aims of legal content management should be the identification of ambiguity of meaning. At least the focus shou= ld be on pointing them out. The resol= ution of ambiguity can then be applied quite naturally as additional characteristics in a given situation tilt the scale to one side or the othe= r. In other words – after the ambiguity has been identified, the user can f= ocus on finding additional units of information that correspond to the facts in order to qualify the issue more accurately.

These challenges are indigenous to legal content; but what do they have= in common? How can they be utilised in improving the usability of legal conten= t? How can they efficiently serve legal content management?<= /p>

One common denominator is their natural presence in any unit of legal content, and this should be the focal point of content research. The general area of legal science is divided into several disciplines interwoven by the following categories relevant to content management:

&nbs= p; &= nbsp; &nbs= p; a) &nb= sp; The creation of legal provisions – the principles and rules of nomotechnics are used to structure individual units= of content with the goal to form a final, fully comprehensive legal document (= e.g. a law or a statute, a judicial decision, a contract, etc.);

&nbs= p; &= nbsp; &nbs= p; b) &nb= sp; The execution of legal provisions – whe= n laws and statutes are applied and the judicial decisions are made the consequenc= es they lead to can be observed in actual circumstances. These in turn usually result in reports – i.e. new documents are created;=

&nbs= p; &= nbsp; &nbs= p; c) &nb= sp; The interpretation of legal provisions – when= new laws and statutes are introduced, and when they come in effect, they freque= ntly need to be interpreted either by the lawmaker(s) or by certain legal profes= sionals. These interpretations as well as other observations made by legal scientists result in even more units of content being created – i.e. commentarie= s, discussions, seminars and conferences, theses and elaborations, etc.

All categories mentioned above provide a practically inexhaustible sour= ce of content, however, at the same time they form an ever growing collection = of documents that need to be properly researched: their contents must be recognised, their informational value assessed, their full potential discov= ered and put to use. The area of legal content management, while being quite bro= ad and diverse, still remains fairly manageable compared to other more general areas of content management.

To sum up – legal content itself provides great possibilities of = use, they just need to be properly researched, analyzed from all relevant aspect= s, and the results of this research used in the development of advanced content management methods, which in turn should be implemented in every-day legal content management. The actual methods of legal content management provide a very interesting subject matter for another lengthy article and so won't be discussed here in detail.

Legal text search

One aspect of content management, however, will be discussed here ̵= 1; the text search, or as we refer to it – the legal text search. As mentioned in the first chapter, sever= al methods of text search are in use today. None of them (by themselves) provi= de an efficient approach to legal text search, which is principally based on t= he meaning of the words and expressions, rather than simply finding documents where these words and expressions are used.

The indigenous characteristics of legal content demand a specialised approach to content management as opposed to the more general approach chos= en by most of today's content and search providers. The efficiency of such specialised ontological methods of content management is also highly plausible based on the following two fact= s:

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; The content itself is highly specialised<= span lang=3DEN-GB style=3D'mso-ansi-language:EN-GB;mso-no-proof:no'>; and

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; The expected use along with target users tends to be highly specialised.

These facts narrow the possibilities down to a manageable quantity R= 11; of course, the demand for investments into the development of such solutions will be considerable. After all, »manageable« does not equal »small«.

At least the development stage of such an undertaking may very well be highly resource-intensive, many new methods of content management would sti= ll need to be developed, even utilising the methods of Artificial Intelligence= [7]= . However, the actual benefits will = be comparably high, and are expec= ted to increase exponentially as the quantity of enhanced content increases.

Once expert knowledge of legal content is materialised as advanced meth= ods of content preparation and content use, the content can be enhanced through the application of these methods. The enhancem= ent will be apparent in all key aspects of legal content management: the preparation, the presentation and the usability of content.

The enhancements will be manifested in the following ways:

= &nb= sp; 1. &nb= sp; Enhanced content:

= &nb= sp; = - &nbs= p; New value is recognised in existing content;

= &nb= sp; = - &nbs= p; New possibilities of content evaluation are created= ;

= &nb= sp; 2. &nb= sp; Enhanced characteristics:

= &nb= sp; = - &nbs= p; New characteristics are discovered;

= &nb= sp; = - &nbs= p; New possibilities of use are created;<= /span>

&nbs= p; &= nbsp; &nbs= p; 3. &nb= sp; Enhanced presentation capabilities:=

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; New methods of content presentation are possibl= e;

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; Elements of special interest (special meaning!) can b= e used either as part of existing units or as new individual units of content;

&nbs= p; &= nbsp; &nbs= p; 4. &nb= sp; Enhanced usability:

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; Text search features are improved by incorpor= ating dynamic guidance;

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; Additional characteristics can be created, such as = ambiguity, disputability and discrepancy warnings.

The above are direct manifesta= tions of enhancements – to speak of indirect manifestations would require another lengthy article or a more broad di= scussion. The impact on users as well as on the authors (editors) would be immense. S= uch advanced tools have the power to improve the productivity of every user in = the area of legal content.

Let's take a look at a few illustrations related to text search:

= &nb= sp; 1. &nb= sp; "interest" – searching for su= ch a general keyword in a regular search engine would produce a large amount of = results, but offer little else.

In the legal text search engine the search would be guided= dynamically:

&nbs= p; &= nbsp; &nbs= p; - &nbs= p; The ambiguity would be resolved first: matches like »interest rate«, »public interest«, = and »interest of investigation« w= ould form dynamic guides of relevance, refining the search results. The user cou= ld continue the search by following the guides, or choose to disregard them and adopt a more ordinary approach;

   &nbs= p;            &= nbsp;           &nbs= p;   2.   &nb= sp; "pontiac= gto" – sea= rching for a more selective keyword in a regular search engine would produce a moderate to large amount of pretty similar results without any additional possibilities.

However, in the legal text search engine the results would include a few enhancements:

   &nbs= p;            &= nbsp;           &nbs= p;       -   &nbs= p; The actual meaning of the keywords would be considered: search resul= ts would be grouped according to common characteristics such as »automobile« (or »motor vehicle«), »motorist liability«, and &ra= quo;hazardous object« to improve= usability. Again, the user could continue the search by following the guide, or choose= to disregard the enhancements and use a less enriched result set;

   &nbs= p;            &= nbsp;           &nbs= p;   3.   &nb= sp; "involuntary manslaughter" – an extremely selective search string would in a regular text search engine pro= duce either a practically indistinguishable result set or even yield no results = at all.

Of course, in the legal text search engine searching for s= uch a keyword would result in:

   &nbs= p;            &= nbsp;           &nbs= p;       -   &nbs= p; A logical set of options: the legal definition would be offered, a collection of judicial cases would be made available, extracts from existing documents would be listed to illustrate r= elevant legal and/or judicial practices, and other relevant keywords would be offer= ed to guide the user. Once again, the user would be able to continue the research= with the enhancements, or choose to disregard them and use a more ordinary result list.

Whenever a user of the legal text search would choose to leave the comf= ort of fully enhanced options, some enhancements would still be available to him – in a less intrusive way. At the same time more enhancements would be made available instantly upon demand. With improved methods of content management enhanced content is always just a cl= ick away.

So, to answer the principal question – why would searching legal texts be any different from searching any other text?

The answer comes in two parts:

      =             &nb= sp;             1.   &nb= sp; Because it could – the content itse= lf provides numerous possibilities, its full potential still ready to be discovered; and

      =             &nb= sp;             2.   &nb= sp; Because it should – the nature of c= ontent and the nature of its use require an advanced approach, and so do the users= of legal documents.

© 2005–2006, Mi Lambda, Matija Lah, s.p. All rights in this document reserved.